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ABSTRACT 

This report describes the methods and procedures used for the 
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included notable changes from previous NPSAS surveys (conducted in 1987, 1990, 
1993, and 1996) in its sample design and collection of data. For example, this 
study is the first to restrict institutional sampling to Title IV 
participating institutions. It is also the first in the NPSAS series to use 
Web-based instrumentation for institutional records collection. However, 
sufficient comparability in survey design and instrumentation was maintained 
to ensure that important comparisons with past NPSAS studied can be made. The 
discussion of methods and procedures contains these chapters: (1) 

"Introduction, Background, and Purpose"; (2) "Design and Method of NPSAS: 
2000"; (3) "Outcomes of Data Collection"; (4) "Evaluation of Operations and 

Data"; (5) Variable Construction and File Development"; and (6) "Weighting and 
Variance Estimation." Eleven appendixes contain supplemental information about 
aspects of the survey. (Contains 133 tables and 11 figures.) (SLD) 
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Foreword 



This report describes the methods and procedures used for the 2000 National Postsecondary 
Student Aid Study (NPSAS:2000). NPSAS:2000 included notable changes from previous 
NPSAS surveys (conducted in 1987, 1990, 1993, and 1996) in its sample design and collection 
of data. For example, the current study is the first to restrict institutional sampling to Title IV 
participating institutions. It is also the first in the NPSAS series to employ web-based 
instrumentation for institutional records collection. However, sufficient comparability in survey 
design and instrumentation was maintained to ensure that important comparisons with past 
NPSAS studies could be made. 

We hope that the information provided in this report will be useful to a wide range of interested 
readers. We also hope that the results reported in the forthcoming descriptive summary reports 
will encourage use of the NPSAS :2000 data. We welcome recommendations for improving the 
format, content, and approach, so that future methodology reports will be more informative and 
useful. 

C. Dennis Carroll 
Associate Commissioner 
Postsecondary Studies Division 
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1 999-2000 National Postsecondary Student Aid 
Study (NPSAS:2000) Methodology Report 



Executive Summary 

Introduction 

The National Postsecondary Student Aid Study (NPSAS), a comprehensive study of 
financial aid among postsecondary education students in the United States and Puerto Rico, 
provides information on trends in financial aid and on the ways in which families pay for 
postsecondary education. NPSAS represents students attending all types and levels of 
institutions, including public, private for-profit, private not-for-profit, less-than-2-year, 2-year, 
and 4-year institutions. The NPSAS data are part of the comprehensive information that the 
National Center for Education Statistics (NCES) provides on student financial aid receipt and 
other characteristics of those enrolled in postsecondary education. 

NPSAS also serves as the base-year survey for longitudinal studies of postsecondary 
students. Thus, NPSAS:2000 was the base-year survey for a sample of baccalaureate degree 
recipients who were interviewed again in 2001. 

This report describes the methods and procedures used for NPSAS:2000. The 
NPSAS:2000 sample design and collection procedures included notable changes from those used 
for previous NPSAS cycles. For example, NPSAS:2000 was the first to restrict institutional 
sampling to institutions having Title IV Program Participation Agreements with the U.S. 
Department of Education. It was also the first to employ a Web-based instrument for collection 
of institutional records. However, sufficient comparability in survey design and instrumentation 
was maintained to ensure that important comparisons with data from previous NPSAS cycles 
could be made. 

Target Population and Sample Design 

The target population for NPSAS: 2000 consisted of all students who were enrolled in 
postsecondary institutions in the United States or Puerto Rico that had Title IV Program 
Participation Agreements with the U.S. Department of Education at any time between July 1, 
1999, and June 30, 2000 (defined as the NPSAS:2000 year). 



The institutional sampling frame for NPSAS:2000 was constructed from the 1998 99 
Integrated Postsecondary Education Data System (IPEDS) Institutional Characteristics (IC) file 
and, because NPSAS:2000 also served as the base-year survey for a longitudinal study of 
baccalaureate recipients, the 1996-97 IPEDS Completions file. Eligible institutions were 
partitioned into 22 institutional strata based on institutional control, highest level of offering, and 
percentage of baccalaureate degrees awarded in education. Approximately 1,100 institutions 
were initially selected for NPSAS :2000, and all but 10 of these institutions were found to be 
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Executive Summary 



eligible. Sampling frames for selecting students consisted of enrollment lists or data files 
provided by the institutions for those students enrolled during the-NPSAS:2000 year. 



The desired number of sample students was determined by accounting for expected rates 
of nonresponse and ineligibility among sample students in different strata and rates of 
misclassification of baccalaureate recipients (as determined from NPSAS:93 and the 
NPSAS:2000 field test). These sampling procedures resulted in the selection of about 70,200 
students for NPSAS:2000, including 16,600 potential baccalaureate recipients. Almost 6,000 of 
these sample members were determined to be ineligible for NPSAS;2000 during various phases 
of data collection, resulting in a final eligible sample of about 64,500 students. 



Data Collection Design and Outcomes 

NPSAS:2000 involved a multistage effort to collect information related to student aid. 

All student sample members were first matched to the U.S. Department of Education’s Central 
Processing System (CPS) to collect an electronic student aid report (Institutional Student 
Information Report, or ISIR) for each federal financial aid applicant. The second stage involved 
abstracting information from the student’s records at the sampled postsecondary institution, using 
a Web-based computer-assisted data entry (CADE) system. Interviews were then conducted with 
sampled students, primarily using a computer-assisted telephone interviewing (CATI) procedure. 
To help reduce the level of nonresponse to CATI, computer-assisted personal interviewing 
(CAPI) procedures, using field interviewers, were also used for the first time on a NPSAS study. 

Over the course of data collection, some data were obtained from the Department of 
Education’s National Student Loan Data System (NSLDS), the ACT and the Educational 
Testing Service. These additional data sources provided information that was not collected from 
the institutions or the students and provided a way to “fill in” institutional record abstraction 
(CADE) data or student interview (CATI) data that were missing for individual sample members 
(e.g., demographic characteristics). The additional data sources also provided a way to check or 
confirm information obtained from student records or the interview. 



Institutional Contacting 

Once institutions were sampled, attempts were made to contact the chief administrator of 
the selected institutions to verify institutional eligibility, solicit participation of eligible 
institutions, and request appointment of an Institutional Coordinator. Coordinators were asked to 
provide lists or data files of all eligible students enrolled in any term within the NPSAS :2000 
year. Several checks on quality and completeness of student lists were implemented before the 
sample students were selected. For applicable schools, separate checks were made for 
baccalaureate recipients, undergraduate students, graduate students, and first-professional 
students. Of the nearly 1,100 eligible institutions, 1,000 provided a student enrollment list or 
data file that could be used for sample selection, for an overall weighted institutional 
participation rate of 95 percent. 
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Institutional Record Abstraction 

A CADE software system was developed for use in collecting data from student records. 
Institutions could choose either to enter the data themselves using a Web-based instrument or to 
have a field data collector enter the data. The CADE instrument was structured into eight 
sections: locating (telephone and address) information, demographic characteristics, admissions 
testing, enrollment, tuition data, financial aid awards, need analysis, and for those students not 
previously matched successfully to the CPS, but who had applied for federal financial aid for the 
study year — ISIR. 

The CADE record abstraction process began when a student sample had been selected 
from an institution’s list and transmitted to the CPS for obtaining financial aid application data. 
Upon completion of the CPS matching, a number of data elements were preloaded into the 
CADE database, thus initializing the CADE system. In addition, the system was customized for 
each institution by preloading the names of up to 10 institution financial aid programs and up to 
10 state financial aid programs. Once CADE was initialized for a particular institution, the 
Institutional Coordinator was notified by telephone that the CADE data collection could begin. 
Institutions that had chosen field data collection were also notified by telephone of CADE 
initialization, at which time an appointment was made for a field data collector to visit the 
institution. 

Records for about 59,300 students (92 percent of the eligible students) were abstracted, 
with almost 70 percent of these abstracted by the institutions themselves using the NPSAS 
CADE Web Site. 

Student Locating and Interviewing 

Using information provided by CADE, sample members were traced to their current 
location prior to conducting the interview using the CATI system. The most current information 
for the student and any other contacts was preloaded into the CATI system to assist the 
interviewers in locating sample members. Cases that were not located during the CATI locating 
process were submitted to the tracing operations unit for intensive locating. Overall, 8 1 percent 
of the eligible sample members were located. 

The CATI system developed for NPSAS:2000 presented interviewers with screens of 
questions to be asked of the respondents, with the software guiding the interviewer and 
respondent through the interview. The student interview consisted of seven sections administered 
sequentially, namely: eligibility, enrollment, financial aid, employment, education experiences 
and expectations, disabilities, and locating information. To reduce interview burden and to guide 
the interview, information collected from CADE and other sources was preloaded before the 
interviews. Online coding programs developed by NCES (for industry/occupation, IPEDS, and 
field of study coding) were embedded in the overall interview administration system. 
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Student interviews were conducted primarily by CATI. A paper-copy mail questionnaire 
or an “abbreviated” telephone interview was also available. All students finalized as 
unbeatable in CATI were eligible for field locating and/or CAPI. Nonresponding and 
unbeatable cases falling within predetermined geographic clusters were assigned to field staff for 
CAPI. CAP I procedures included attempts to locate, gain cooperation from, and interview 
sample members either by telephone or in person. Similar cases not in an identified cluster were 
assigned to field locators. Field locators then attempted to locate the students and convince them 
to call an 800 number to complete the interview in CATI. 

Of the eligible sample members located, about 44,500 (87 percent) were interviewed. 
Adjusting for institution nonresponse, the overall weighted CATI response rate was 66 percent. 
Ninety-one percent of those interviewed completed the full interview. 

Study Respondents 

Students included in the final NPSAS:2000 analysis file were those students with 
completed institutional records (CADE) data and/or completed student interview (CAPI or 
CATI) data. Using this definition, about 61,800 of the 64,500 eligible sample students were 
classified as study respondents, for an unweighted student yield of 96 percent. After adjusting 
for institutional nonresponse and for attendance at more than one institution, the overall weighted 
study response rate was 89 percent. 

Evaluation of Operations and Data Quality 

Evaluations of NPSAS:2000 operations and procedures focused on the time line for data 
collection, the effectiveness of student tracing and locating procedures, refusal conversion 
efforts, the use of incentives for selected respondent groups, and the length of the student 
interview. Evaluations of data quality included analysis of nonresponse bias, examination of 
items with high rates of “don’t know” and “refusal” responses, interviewer use of online help 
text, item coding and administration errors, quality control procedures, and analysis of the 
stability of item responses over time. 

Data Files 

Data are available for the 61,800 study respondents, including about 49,900 
undergraduate students, 10,600 graduate students, and 1,200 first-professional students. 

Statistical analysis weights adjusting for unequal sampling rates and differential propensities to 
respond were computed for respondents. 
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'Products 

NPSAS:2000 reports or data products that have been or will be published include the 
following: • - 

National Postsecondary Student Aid Study: Student Financial Aid Estimates for 
] 999-2000 (NCES 2001-209). http://nces.ed.gov/pubsearch/pubsinfo.asp7pubkH2001209 This 
report briefly describes key findings from NPSAS:2000. 

Profile of Undergraduates in U.S. Postsecondary Education Institutions: 1999—2000 
(NCES 2002-168) http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2002168 . This report 
contains detailed tables on the characteristics of undergraduates enrolled during 1999-2000, 
including age, race/ethnicity, gender, income, financial aid receipt, community service, veteran 
status, and more. It also includes an essay on the diversity of undergraduate students. 

Student Financing of Undergraduate Education: 1999-2000. (NCES 2002-167) 
http://nces.ed.gov/pubsearch/pubsinfo.asp7pubkH20Q2167 . This report focuses on how 
undergraduate students enrolled during 1999—2000 financed their education, providing detailed 
tables on the distribution and average amounts of grants, loans, and work-study funds received by 
students from federal, state, institutional, and private sources. These data are shown by selected 
student characteristics, such as age, gender, race/ethnicity, income, and attendance status for the 
various types of institutions. Information includes tuition, total student budgets, and the net price 
of attendance by type of institution. The report also includes an essay on students who borrow at 
the federal loan limits. 

Student Financing of Graduate and First-Professional Education: 1999—2000 (NCES 
2002-166 http://nces.ed. gov/pubsearch/pubsinfo.asp?pubid=2002 166 . This report describes the 
characteristics of graduate and first-professional students enrolled during 1999—2000, including 
age, race, gender, income, financial aid receipt, community service, veteran status, and more. It 
also describes those graduate and first-professional students who received financial aid 
including grants, loans, and work-study — from federal, state, institutional, or other sources, by 
selected student characteristics. In addition, the report includes an essay on graduate students 
with assistantships. 

NPSAS:2000 Undergraduate and Graduate/First Professional Data Analysis Systems. 
These Windows-based software applications provide public access to the NPSAS:2000 survey 
data. Users can generate tables of percentages, means, or correlation coefficients by choosing the 
Data Analysis System (DAS) variables of interest and specifying what function should be used. 

NPSAS. 2000 Restricted-Use Electronic Codebook and Data Files. This data product 
provides the complete data obtained through NPSAS:2000, documented by the electronic 
codebook (ECB). It is available only to researchers who have applied for and received 
authorization from NCES to access restricted-use research files. Contact Cynthia Barton, Data 
Security Officer, at 202—502—7307, or e-mail Cvnthia.Barton@ed.gov . 
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Chapter 1 

Introduction, Background, and Purpose 

This document describes the methodological procedures and results for the 2000 National 
Postsecondary Student Aid Study (NPSAS:2000). NPSAS:2000 is a comprehensive study of 
financial aid among postsecondary education students in the United States and Puerto Rico, and 
provides current information on how families pay for postsecondary education. The study was 
conducted for the National Center for Education Statistics (NCES) of the U.S. Department of 
Education (ED), as authorized by Title IV, Section 401 of the National Education Statistics Act 
of 1994 (P.L. 103-382). NPSAS:2000 was conducted under contract by Research Triangle 
Institute (RTI), assisted by MPR Associates, Inc., and the National Association of Student 
Financial Aid Administrators (NASFAA). 

This introductory chapter describes briefly the background, purposes, schedule, and 
products of the NPSAS:2000 full-scale study. The study design, sampling and data collection 
procedures are described in Chapter 2. The third chapter describes the overall outcomes for the 
several stages and sources of data collection. Chapter 4 examines the effectiveness of the 
procedures and methodologies employed in the study, and data quality issues. Chapter 5 
describes the details of data editing, processing, and file development operations. Chapter 6 
summarizes the NPSAS:2000, weighting and variance estimation activities. 

Materials used during NPSAS:2000 data collection are provided as appendices to the 
report. These include: a list of the experts comprising the NPSAS:2000 technical review panel 
(appendix A); materials sent to institutions and students, as well as endorsements obtained from 
professional organizations and associations in support of the study (appendices B and C); 
contents of training materials (appendix D); and facsimiles of the study’s data collection 
instruments (appendices E and F). Additional appendices provide supporting documentation 
regarding details of the complex sampling design developed for the study (appendix G), 
supplemental tables and design effects (appendices H and I), analysis variables (appendix J), and 
imputations (appendix K). 

1.1. Background and Purpose of NPSAS 

NPSAS is a comprehensive nationwide study designed to determine how students and 
their families pay for postsecondary education, and to describe some demographic and other 
characteristics of the students enrolled in postsecondary education. The study is based on a 
nationally representative sample of students in postsecondary education institutions, including 
undergraduate, graduate, and first-professional students. Students attending all types and levels 
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of institutions are represented, including public and private for-profit and not-for-profit 
institutions, and less-than-2-year institutions to 4-year colleges and universities. The NPS AS 
studies are designed to address the policy questions resulting from the rapid growth of financial 
aid programs and the succession of changes in financial aid program policies since 1986. The 
first NPS AS study was conducted in 1986-87; subsequent studies have been carried out during 
the 1989-90, 1992-93, and 1995-96 school years (i.e., NPSAS:90, NPSAS:93, and NPSAS:96). 
This methodology report relates to the latest study in this series, NPSAS:2000, for which data 
were collected from sample students enrolled between July 1 999 and June 2000. 

In addition to collecting information on financial aid in the United States, since 1990 
NPSAS has been used to form the base-year sample for a postsecondary longitudinal survey 
supported by NCES. Specifically, alternate NPSAS data collections provide the base year 
sample for either the Beginning Postsecondary Students (BPS) longitudinal study or the 
Baccalaureate and Beyond (B&B) longitudinal study. NPSAS :2000 serves as the base-year 
survey for a sample of baccalaureate students who will be surveyed again in 2001 . 

A main objective of the NPSAS study is to produce reliable national estimates of 
characteristics related to financial aid for postsecondary students. The data are part of NCES' 
comprehensive information on student financial aid and other characteristics of those enrolled in 
postsecondary education. The study focuses on three general questions with important policy 
implications for financial aid programs: 

• How do students and their families finance postsecondary education? 

• How does the process of financial aid work, in terms of both who applies for and who 
receives aid? 

• What are the effects of financial aid on students and their families and on 
postsecondary institutions? 

1.2. Methodological Issues 

As described in Chapter 2, the NPSAS survey design is both large and complex. Data are 
collected from a very large and diverse set of students. A major methodological concern 
underlying NPSAS is selecting data sources that provide some assurance of comparability for 
each element. Of the potential sources for NPSAS data — government data files, institutional 
records, and students — none alone can provide a complete and accurate summary of 
postsecondary education financing. 

Financial aid offices maintain accurate records of certain types of financial aid at that 
institution, but these records are not necessarily inclusive of all support and assistance. Such 
records may not contain financial aid provided at other institutions attended by the student or 
those not recorded by a financial aid office. 1 Students and their parents are more likely than 
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institutions to have a comprehensive picture of education financing, but may not have accurate 
memory or records of exact amounts and sources. They may have provided information to 
lending agencies or aid providers (or clearinghouses), and that information may exist in student 
financial aid records. Consequently, the NPSAS data requirements call for a survey design that 
builds a comprehensive and accurate understanding of postsecondary education financing from a 
number of different sources. To meet this challenge, NPSAS:2000 relied on an integrated system 
of computer-assisted data capture instruments. 

Innovative methodological solutions that were applied to NPSAS :2000 challenges were 
tested and refined during a substantial field test conducted during the 1998-99 school year on a 
separate independent sample of students and institutions. Results of the field test have been 
reported separately. 2 



1.3. Special Features of NPSAS:2000 

Although the general purposes of the NPSAS studies have remained quite consistent, all 
NPSAS implementations except the first also have served as the base year for a longitudinal 
study. For NPSAS:96 and NPSAS:90, the longitudinal cohort comprised students who began 
their postsecondary education during the NPSAS year. NPSAS :2000 and NPSAS: 93 have 
provided the base-year cohort for a sample of students who completed a baccalaureate degree 
during the NPSAS year. As in the past, the NPSAS :2000 longitudinal cohort was oversampled 
to support the subsequent longitudinal follow-up study. 

In implementing four prior rounds of NPSAS and their associated field tests, NCES and 
its contractors have developed and refined a number of systems and methods to facilitate 
subsequent rounds. Consequently, in NPSAS :2000, most methods that both had proved 
successful and remained applicable to current study needs were maintained or refined. Like prior 
NPSAS implementations, however, the current study also attempted to take advantage of new 
technologies and to access newly available data sources toward improving study efficiency 
and/or the quality of data collected. 

The most significant enhancement to NPSAS :2000 involved the student record 
abstraction process. For NPSAS :2000, a new computer-assisted data entry (CADE) system for 
use over the Internet through the World Wide Web was developed and implemented. This Web- 
based software (Web-CADE) had a better user interface than the NPSAS :96 system, and 
addressed several of the self-CADE issues raised during the previous study (insufficient 
computer memory, failures during diskette installation and virus scanning, lack of information 
regarding institutions’ progress during data collection). 

NPSAS :2000 continued procedures implemented in 1996 to broaden the base of 
postsecondary student types for whom telephone interview data could be collected. In past 

2 For results of the NPSAS:2000 field test, which tested procedures and instruments before the start of the 
full-scale study, see U.S. Department of Education, National Center for Education Statistics. National Postsecondary 
Student Aid Study ( NPSAS. 2000 ) Field Test Methodology Report, NCES No. 2000-17, by Melissa R. Biber, Michael 
W. Link, John A. Riccobono, and Peter H. Siegel. Andrew G. Malizio, project officer. Washington, DC: October 
2000. 
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NPSAS implementations, no mechanism existed for contacting and collecting information by 
telephone from students with severe hearing impairments; however, both NPSAS :96 and 
NPSAS :2000 included the use of Telephone Display for the Deaf (TDD) technology to facilitate 
telephone communications with such students. Also, beginning in NPSAS :96, a separate 
Spanish translation interview was prepared for administration to students who had insufficient 
English language proficiency to complete the interview in English or who needed at least some 
translation of terms by a bilingual interviewer. 3 This accommodation was particularly useful 
with the students from sampled postsecondary institutions in Puerto Rico. 



1.4. Overall Schedule and Products of NPSAS:2000 

Table 1-1 includes a schedule of activities for the NPSAS :2000 study. As noted 
previously, the NPSAS:2000 full-scale study was preceded by a field test, and data collection for 
the full-scale study spanned the 1 1 -month period from March 2000 to February 2001. 



Table 1-1. — Start and end dates for major NPSAS:2000 activities 



Activity 


Start 

date 1 


End 

date 2 


Select institutional sample 


10/28/99 


01/02/99 


Make mail and phone contact with chief administrator 


11/24/99 


12/15/00 


Make mail and phone contact with institutional coordinator 


01/05/00 


12/04/00 


Obtain lists for student sampling 


02/17/00 


12/13/00 


Select student samples 


02/17/00 


12/13/00 


Request/obtain 1999-2000 data from the Central Processing System (CPS) 


02/18/00 


12/20/00 


Preload CPS data into CADE records 


03/20/00 


02/01/01 


Implement CADE record abstraction 


03/23/00 


02/16/01 


Preload CADE into computer-assisted telephone interviewing (CATI) records 


05/19/00 


02/16/01 


Implement CATI 


05/22/00 


02/28/01 


Request/obtain 2000-2001 CPS data 


01/12/01 


01/12/01 


Request/obtain 1999-2000 Pell Grant data 


01/05/00 


01/22/01 



'This is the date on which the activity was initiated for the first applicable school and/or its associated students. 
2 This is the date on which the activity was completed for the last applicable school and/or its associated students. 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Postsecondary Student Aid Study, 
1999-2000 (NPSAS:2000). 



3 It was expected that Spanish would be the primary language for the largest non-English speaking segment 
of the sampled population. English/Spanish bilingual interviewers were used in NPSAS:2000 as in previous NPSAS 
studies to interview Spanish-speaking sample members with limited English proficiency. Cost considerations 
precluded similar accommodations for other foreign languages. 
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The following products/reports based on NPSAS:2000 will be available in 2002: 

Undergraduate Financing of Postsecondary Education, 1999-2000 

This report will focus solely on undergraduate students enrolled during the 1999-2000 
school year. It will examine how undergraduate students financed their education. The report 
will have a section that explores undergraduate borrowing, including information from the 
National Student Loan Data System on cumulative borrowing. Other tables in the report will 
summarize total price of attendance, the distribution of financial aid among students by type of 
institution, and the net price of attendance. This report will contain a special section presenting 
the distribution of aid among students at different types of institutions with a focus on student 
borrowing. Supplemental tables for students who borrow at the Stafford loan limit will also be 
included. 

Student Financing of Graduate and Professional Education, 1999-2000 

This report will describe the characteristics of graduate and first-professional students 
enrolled during 1 999-2000, including age, race, gender, income, community service, veteran 
status, and more. Also, the report will describe those graduate and first-professional students 
who received financial aid, including grants, loans, and work-study from federal, state, 
institution, or other sources, by selected student characteristics. The report will include a section 
on graduate research and teaching assistantships. 

Profile of Undergraduates at U.S. Postsecondary Institutions, 1999-2000 

The profile will describe the characteristics of undergraduates enrolled during 
1 999-2000, including age, race, gender, income, financial aid receipt, community service, 
veteran status, and student employment. It will include a special section highlighting the 
diversity of the undergraduate population, focusing on demographic composition, race/ethnicity, 
immigration status, and undergraduates with dependents. 

NPSAS. 2000 Undergraduate and Graduate/First-Professional Data Analysis Systems 

The Data Analysis System (DAS) is a Windows-based software application that provides 
public access to NCES survey data. Two DASs have been created from the NPSAS:2000 data: 
an undergraduate DAS and a graduate/first-professional DAS. With the DAS, users can generate 
tables of percentages, means, or correlation coefficients simply by choosing the DAS variables 
(based on survey questionnaire items) that they would like to appear in a table and indicating 
what function should be used. 
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Contact Aurora D’Amico, or visit the website (http://nces.ed.gov/das/) to download a 
NPSAS:2000 DAS application or one of the NPSAS:2000 reports. 

Aurora D’Amico 

Postsecondary Coop System, Analysis & Dissemination- 

Postsecondary Studies Division 

Phone: (202) 502-7334 

E-mail: Aurora.D’ Amico@ed.gov 

NPSAS. 2000 Restricted use data files 

The survey data files used to create variables in the Data Analysis Systems, and the 
associated electronic codebooks and file documentation, are available to researchers who have 
obtained a restricted data license from NCES. Information on obtaining a restricted data license 
may be found in the NCES Restricted Use Data Procedures Manual 4 , available from Cynthia 
Barton. 



Cynthia L. Barton 
Data Security Officer 
Phone: (202) 502-7307 
E-mail: Cvnthia.Barton@ed.gov 

Information on the NCES Statistical Standards Program, including Restricted Use Data 
Licenses Procedures, is available from the NCES website: http://nces.ed.gov/statprog. 



4 U.S. Department of Education. National Center for Education Statistics. NCES Restricted-Use Data 
Procedures Manual. Washington, DC: October 1999. 
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Chapter 2 

Design and Method of NPSAS:2000 



Implementation of NPSAS:2000 required a complex set of study tasks and activities, 
including sequentially dependent data collection operations as well as overlapping development, 
analysis, documentation and reporting tasks. An RTI-developed Integrated Management System 
(IMS), used effectively in other large-scale survey projects, was adapted, based on results of an 
extensive NPSAS field test, 1 for use in the full-scale NPSAS:2000. 

2.1 NPSAS:2000 Target Population and Sampling Overview 

The basic features of the NPSAS:2000 sampling plan and the resulting samples are 
summarized in the sections 2.1.1 and 2.1.2. Greater detail is provided in appendix G for the 
interested reader. 



2. 1. 1 Target Population 

The target population for NPSAS:2000 consisted of all students enrolled at any time in 
postsecondary institutions in the United States or Puerto Rico and which had signed Title IV 
participation agreements with the U.S. Department of Education making them eligible for the 
federal student aid programs (Title IV institutions) between July 1, 1999, and June 30, 2000. 2 
With one exception, the survey population also was defined as those students who were enrolled 
at any time between July 1, 1999, and June 30, 2000. The exception occurred if a term or course 
began after May 31, 2000, and ended after June 30, 2000, then students enrolled only in that term 
or course were excluded from the survey population. 3 This definition of the survey population 
differed from previous NPSAS rounds but was more consistent with the definition of the target 
population. More specific definitions of the institution and student populations are provided in 
section 2.2. 



Though NPSAS:2000 was limited to Title IV institutions, prior NPSAS rounds also 
surveyed students enrolled at institutions not participating in Title IV aid programs. In addition. 



'U.S. Department of Education. National Center for Education Statistics. National Postsecondary Student 
Aid Study (NPSAS:2000) Field Test Methodology Report, NCES No. 2000-17, by Melissa R. Biber, Michael W. 
Link, John A. Riccobono, and Peter H. Siegel. Andrew G. Malizio, project officer. Washington, DC: October 



2000. 



2 Excluding students who were enrolled in military service academies, were enrolled solely in a General 
Education Development (GED) program, or were concurrently enrolled in high school. 

3 The target population is the population about which inferences will be made. The survey population is the 
population actually covered by the sampling frame. Nearly all members of the target population were also members 
of the survey population; however, the adopted definition of the survey population allowed the student lists needed 
for sample selection to be obtained before or during June for many institutions (e.g., those on a semester calendar 
system). Poststratification adjustments of the analysis weights (see Chapter 6) reduce any resulting bias for 
inferences regarding the target population. 
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for NPSAS:96 and NPSAS:93, the survey population was defined as those students who were 
enrolled in any term beginning between May 1 and April 30 during the survey year, i.e., 1995-96 
and 1992-93, respectively; for NPSAS:90, the students sampled were those enrolled on 
August 1, 1989; October 15, 1989; February 15, 1990; or June 15, 1990 (however, the June 15 
enrollees were not sampled for 4-year institutions because of budgetary limitations); for 
NPSAS:87, only fall 1986 enrollees were sampled. 

2.1.2 Sample Design Overview 

An overview of the sequential statistical sampling process for NPSAS:2000 is provided 
in figure 2-1. The institutional sampling frame for NPSAS:2000 was constructed from the 1998— 
99 Integrated Postsecondary Education Data System Institutional Characteristics (IPEDS-IC) file 
and, because NPSAS:2000 also served as the base-year survey for a Baccalaureate and Beyond 
longitudinal study, the 1996-97 IP EDS completion file was used to check frame information 
regarding estimated size of institutional graduating classes. Both of these files were the latest 
available at the time of NPSAS institutional sampling. 

Figure 2-1. — Schematic of sequential NPSAS :2000 sampling operations 




SOURCE: U.S. Department of Education, National Center for Education Statistics. National Postsecondary Student Aid Study, 
1999-2000 (NPSAS:2000). 
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The IPEDS-IC database provided nearly complete coverage of the institutions in the 
target population. Listings in the file that were not eligible institutions (e.g., institutions located 
outside the U.S. and Puerto Rico; central offices; military academies) were deleted from the 
population file. Additional information for eligible institutions was obtained from the 1996-97 
IPEDS completion files. The eligible institutions were then partitioned into 22 institutional strata 
based on institutional control, highest level of offering, and percentage of baccalaureate degrees 
awarded in education: 

1. Public less-than-2-year 

2. Public 2-year 

3. Public bachelor’s high education 4 

4. Public bachelor’s low education 

5. Public master’s high education 

6. Public master’s low education 

7. Public doctorate-granting high education 5 

8. Public doctorate-granting low education 

9. Public first-professional-granting high education 

10. Public first-professional-granting low education 

1 1 . Private not-for-profit less-than-2-year 

12. Private not-for-profit 2-year 

13. Private not-for-profit bachelor’s high education 

14. Private not-for-profit bachelor’s low education 

15. Private not-for-profit master’s high education 

16. Private not-for-profit master’s low education 

17. Private not-for-profit doctorate-granting high education 

18. Private not-for-profit doctorate-granting low education 

19. Private not-for-profit first-professional-granting high education 

20. Private not-for-profit first-professional-granting low education 

21. Private for-profit less-than-2-year 

22. Private for-profit 2-year or more 

A stratified sample of 1,082 institutions was then selected with probabilities proportional 
to size (pps); some of these institutions subsequently proved to be ineligible and others failed to 
participate. The sampling frames for selecting sample students were paper-copy and electronic 
lists of students provided by the sample institutions for those students enrolled in terms or 
courses of instruction during the previously defined NPSAS year. 6 Student lists were sampled on 
a flow basis as they were received, using stratified systematic sampling. The seven student 
sampling strata were as follows: 



4 For each category that had a high education and low education breakout, the high education stratum was 
defined to be the 20 percent of institutions with the highest proportions of their baccalaureate degrees awarded in 
education (based on the 1996-97 IPEDS completions file). The remaining 80 percent constituted the low education 
stratum. The purpose of this stratification was to ensure a certain sample size of students going into the teaching 
profession which is an important analysis domain for the baccalaureate and beyond longitudinal study. 

5 Institutions that awarded first-professional degrees were included in the doctorate-granting stratum. 

6 Quality control checks were performed on each list received from a sample institution, by comparing the 
numbers of undergraduate, graduate, and first-professional students listed to the “unduplicated” head counts reported 
for the 1997-98 academic year in the 1998-99 IPEDS-IC file. The number of baccalaureates listed was compared to 
the counts reported for the 1996-97 academic year in the 1996-97 IPEDS completions file. 
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1 . Students receiving a baccalaureate degree in business 7 

2. Other baccalaureate recipients 

3. Other undergraduate students 

4. Master’s students 

5. Doctoral students 

6. Other graduate students 

7. First-professional students 

The list for each student stratum was sampled at a rate designed to provide approximately 
equal student-level probabilities. Student sampling rates were revised after enough lists had been 
received to more accurately estimate the overall sample yield. These sampling procedures 
resulted in selection of 70,232 students. 

2.2 NPSAS:2000 Sample Implementation 

The goal of all sampling activities was to attain the targeted numbers of eligible sample 
postsecondary students within each of the specified student and institution strata. An important 
domain of the student sample was the set of students identified as baccalaureates, 8 who are the 
baseline cohort for the Baccalaureate and Beyond (B&B) longitudinal study. The desired number 
of sample students was determined by accounting for expected (from prior NPSAS rounds) rates 
of nonresponse and ineligibility among sample students and rates of B&B misclassification (as 
determined from NPSAS:93 and the NPSAS:2000 field test). Since the student samples were 
selected on a flow basis as sample institutions provided their enrollment lists in order to meet the 
data collection schedule, the students were sampled at fixed rates. For each institution, these 
rates were set based on the institution’s probability of selection and the overall student stratum 
sampling rates. The sampling rates were set to meet or exceed the sample sizes shown in 
table 2-1. 

The NPSAS:2000 sample was also designed to obtain at least 30 student CATI 
respondents from each sample institution that had at least that many eligible students enrolled 
during the NPSAS year. Consequently, institution sample sizes were determined to achieve an 
average of approximately 40 or more sample students per institution within each institutional 
stratum. Given these student sample size goals, the desired number of participating institutions 
was determined to be 1,008. 9 Based on institutional participation rates obtained in prior NPSAS 
rounds and the NPSAS :2000 field test, an initial sample of 1,082 institutions was selected. 



7 Students receiving a baccalaureate degree in business were in a separate stratum so that they would be 
selected at a lower sampling rate than other baccalaureate recipients, because sampling them at the same rate would 
result in more students receiving a baccalaureate degree in business than desired. 

8 Students who received their bachelor’s degree during the 1999-2000 academic year. 

9 An institution was considered participating if it sent in a usable enrollment list. 
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Table 2-1. — Target numbers of sample students, by institutional stratum and type of 
student 



Institutional stratum 


Total 


Baccalaureate 


Other 

undergraduate 


Graduate 


First- 

professional 


Total 


70,266 


16,372 


40,918 


11,657 


1,319 


Public 












1 Less-than-2-year 


1,996 


t 


1,996 


t 




2 2-year 


10,976 


t 


10,976 


t 


t 


Total less-than-4-year 


12,972- 


t 


12,972 




t 


3 Bachelor’s high education 


236 


127 


109 


t 


t 


4 Bachelor’s low education 


923 


175 


740 


t 


t 


5 Master’s high education 


2,124 


1,223 


694 


208 


t 


6 Master’s low education 


6,640 


1,970 


3,636 


1,042 


t 


Total 4-year non-doctorate-granting 


9,924 


3,495 


5,180 


1,249 


t 


7 Doctorate-granting high education 


2,371 


1,229 


719 


423 


t 


8 Doctorate-granting low education 


5,884 


f496 


2,702 


1,686 


t 


9 First-professional-granting high education 


3,985 


1,983 


1,175 


764 


63 


10 First-professional-granting low education 


9,900 


2,677 


4,021 


2,776 


427 


Total 4-year doctorate-granting 


22,141 


7,386 


8,617 


5,648 


490 


Private not-for-profit 












1 1 Less-than-2-year 


601 


t 


601 


t 


t 


1 2 2 -year 


1,201 


t 


1,201 


t 


t 


Total less-than-4-year 


1,802 


t 


1,802 


t 


t 


13 Bachelor’s high education 


739 


423 


315 


t 


t 


14 Bachelor’s low education 


1,586 


583 


999 


t 


t 


15 Master’s high education 


1,595 


855 


543 


197 


t 


1 6 Master’s low education 


3,655 


1,049 


1,800 


810 


t 


Total 4-year non-doctorate-granting 


7,574 


2,910 


3,658 


1,006 


t 


17 Doctorate-granting high education 


781 


263 


209 


309 


t 


1 8 Doctorate-granting low education 


1,310 


262 


418 


630 


t 


19 First-professional-granting high education 


3,216 


959 


1,054 


994 


210 


20 First-professional-granting low education 


4,013 


956 


856 


1,589 


612 


Total 4-year doctorate -granting 


9,320 


2,439 


2,538 


3,521 


822 


Private for-profit 












2 1 Less-than-2-year 


4,328 


t 


4,328 


t‘ 


t 


22 2-year or more 


2,203 


141 


1,823 


232 


7 


Total private for-profit 


6,531 


141 


6,151 


232 


7 



fNot applicable. 

NOTE: “High education” refers to the 20 percent of institutions with the highest proportions of their baccalaureate degrees 
awarded in education (based on the 1996-97 IPEDS completions file). The remaining 80 percent of institutions were classified 
as "low education" (i.e., having a lower proportion of baccalaureate degrees awarded in education). 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Postsecondary Student Aid Study, 
1999-2000 (NPSAS:2000). 

2.2.1 Institutional Sample 

The target population for NPSAS:2000 included nearly all Title IV participating 
postsecondary institutions in the 50 states, the District of Columbia, and Puerto Rico. 10 



10 Title IV participating institutions excluded from the target population were the five U.S. service 
academies. 
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To be eligible for NPSAS:2000, an institution was required, during the 1999-2000 academic 
year, to: 11 

• offer an educational program designed for persons who had completed secondary 
education; 

• offer more than just correspondence courses; 

• offer at least one academic, occupational, or vocational program of study lasting at 
least 3 months or 300 clock hours; 

• offer courses that were open to more than the employees or members of the company 
or group (e.g., union) that administered the institution; 

• be located in the 50 states, the District of Columbia, or Puerto Rico; 

• be other than a U.S. Service Academy; 12 and 

• have a signed Title IV participation agreement with the U.S. Department of 
Education. 

As indicated above, institutions providing only avocational, recreational, or remedial courses or 
only in-house courses for their own employees were excluded. 

The student sample was allocated to the separate applicable institutional and student 
sampling strata, defined above. Student sampling rates, which were used to compute institution- 
level composite measures of size, were based on 1998-99 IPEDS IC and 1996-97 IPEDS 
completions file counts and the required sample sizes (see appendix G for details). 

An independent sample of institutions was selected for each institutional stratum using 
Chromy’s 13 sequential probability minimum replacement (pmr) sampling algorithm to select 
institutions with probabilities proportional to their computed measures of size. However, rather 
than multiple selections of sample institutions being allowed, 14 those with expected frequencies 
of selection greater than unity (1 .00) were selected with certainty. The remainder of the 
institutional sample was selected from the remaining institutions within each stratum. The 
sampling algorithm was implemented with a random start for each institutional stratum to ensure 
the positive pairwise probabilities of selection that were needed for proper variance estimation. 15 



"The listed eligibility requirements are consistent with those used in previous NPSAS rounds, except for 
the last one. 

l2 These academies were not eligible for this fmancial aid study because of their unique funding/tuition base. 

13 J.R. Chromy. “Sequential Sample Selection Methods.” Proceedings of the American Statistical 
Association Section on Survey Research Methods of the American Statistical Association , 1 979, 40 1-406. 

14 Precluding institutions with multiple selections at the first stage of sampling made it unnecessary to select 
multiple second-stage samples of students. 

15 J.R. Chromy (1981). Variance Estimators for a Sequential Sample Selection Procedure. In. D. Krewski, 
R. Platek, and J.N.K. Rao (Eds.), Current Top IMS in Survey Sampling (pp. 329-347). New York: Academic Press. 
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; The numbers of certainty and noncertainty schools selected, within each of the 22 
institutional strata, are shown in table 2-2. Within each institutional stratum, additional implicit 
stratification was accomplished by sorting the stratum sampling frame in a serpentine manner. 

For less-than-2-year, 2-year, and private for-profit institutions, the implicit strata were: (1) 
institutional level of offering (where levels had been collapsed to form strata); (2) the OBE 
Region from the EPEDS IC file (Bureau of Economic Analysis of the U.S. Department of 
Commerce Region); 17 (3) the Federal Information Processing Standard (FIPS) state code; and (4) 
the institution measure of size. For public 4-year and private not-for-profit 4-year institutions, 
the implicit strata were: (1) Carnegie classifications of postsecondary institutions or groupings of 
Carnegie classifications; (2) historically black colleges and universities (HBCU) indicator; 

(3) the Region from the EPEDS-IC file; and (4) the institution measure of size. The objectives of 
this additional, implicit stratification were to approximate proportional representation of 
institutions on these measures. Table 2-3 shows that the regional distribution of the sample is 
consistent with the sampling frame. 

2.2.2 Student Sample 

The postsecondary students eligible for NPSAS:2000 were those who attended a NPSAS- 
eligible institution during the 1999-2000 academic year and who were 

• enrolled in either (1) an academic program; (2) at least one course for credit that 
could be applied toward fulfilling the requirements for an academic degree; or (3) an 
occupational or vocational program that required at least 3 months or 300 clock hours 
of instruction to receive a degree, certificate, or other formal award; 

• not concurrently enrolled in high school; and 

• not enrolled solely in a GED or other high school completion program. 

Each sampled institution that was verified as NPSAS-eligible was asked to provide lists 
of all its students who satisfied all theNPSAS eligibility conditions, preferably “unduplicated” 
(i.e., each student’s name appeared only once) electronic lists (sent via e-mail, diskette, CD- 
ROM, or file transfer protocol [FTP]), together with identifying and classifying information (see 
Section 2.2.3. below). Although electronic files were preferred, the preferences of sample 
institutions were accommodated, and whatever type(s) of student list(s) they were able to provide 
were accepted, as long as they were complete. Separate, “unduplicated” lists were requested for 
baccalaureate business, baccalaureate nonbusiness, other undergraduate (i.e., non-baccalaureate 
undergraduates), master’s, doctoral, other graduate, and first-professional students (the sampling 
strata) from institutions providing paper-copy lists. As expected, however, many institutions 



16 R.L Williams, and J.R Chromy. “SAS Sample Selection MACROS.” Proceedings of the Fifth Annual 
SAS Users Group International Conference , 1980, 392-396. 

17 For sorting purposes, Alaska and Hawaii were combined with Puerto Rico in the Outlying Areas region 
rather than in the Far West region. 
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Table 2-2. — Institutional sampling rates and number of certainty and noncertainty 
institutions sampled, by institutional stratum 



Institutional stratum 1 


Size of 
universe 2 


Sampling 

Rate 


Number of sample institutions 


Total 3 


Certainty 


Noncertainty 


Total 


6,422 


0.17 


1,082 


286 


796 


Public 












1 Less-than-2-year 


255 


0.14 


34 


8 


26 


2 2-year 


1,208 


0.16 


198 


9 


189 


Total less-than-4-year 


1,463 


0.16 


232 


17 


215 


3 Bachelor’s high education 


18 


0.29 


5 


0 


5 


4 Bachelor’s low education 


69 


0.27 


19 


1 


18 


5 Master’s high education 


51 


0.49 


25 


2 


23 


6 Master’s low education 


196 


0.40 


78 


6 


72 


Total 4-year non-doctorate-granting 


334 


0.38 


127 


9 


118 


7 Doctorate-granting high education 


25 


1.00 


25 


25 


0 


8 Doctorate-granting low education 


82 


0.77 


63 


31 


32 


9 First-professional-granting high education 


29 


1.00 


29 


29 


0 


10 First-professional-granting low education 


115 


0.89 


103 


88 


15 


Total 4-year doctorate-granting 


251 


0.88 


220 


173 


47 


Private not-for-profit 












1 1 Less-than-2-year 


112 


0.10 


12 


0 


12 


12 2-year 


314 


0.07 


23 


2 


21 


Total less-than-4-year 


426 


0.08 


35 


2 


33 


13 Bachelor’s high education 


112 


0.15 


17 


0 


17 


14 Bachelor’s low education 


402 


0.09 


37 


0 


37 


1 5 Master’s high education 


120 


0.31 


37 


0 


37 


1 6 Master’s low education 


414 


0.20 


82 


6 


76 


Total 4-year non-doctorate-granting 


1,048 


0.16 


173 


6 


167 


1 7 Doctorate-granting high education 


24 


0.66 


16 


7 


9 


1 8 Doctorate-granting low education 


88 


0.31 


27 


4 


23 


19 First-professional-granting high education 


80 


0.71 


57 


32 


25 


20 First-professional-granting low education 


294 


0.23 


68 


34 


34 


Total 4-year doctorate-granting 


486 


0.35 


168 


77 


91 


Private for-profit 












21 Less-than-2-year 


1,386 


0.06 


77 


0 


77 


22 2 -year or more 


1,028 


0.05 


50 


2 


48 


Total private for-profit 


2,414 


0.05 


127 


2 


125 



Stratum reflects institutional categorization as determined from the 1998—99 IPEDS IC file; some errors in this classification 
were uncovered when institutions were contacted. 

2 Based on the 1998-99 IPEDS IC file. 

During institutional contacting, it was discovered that part of one school had recently split off and formed a separate institution. 
Both institutions were included in the sample, adding another institution to stratum 10, so the actual total sample size is 1,083. 

NOTE: High education” refers to the 20 percent of institutions with the highest proportions of their baccalaureate degrees 

awarded in education (based on the 1996-97 IPEDS completions file). The remaining 80 percent of institutions were classified 
as “low education” (i.e., having a lower proportion of baccalaureate degrees awarded in education). 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Postsecondary Student Aid Study 
1 999-2000 (NPS AS:2000). 
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Table 2-3.— Distribution of NPSAS:2000 institutional sample, by region 



Region* 


Sample institutions 


IPEDS institutions 2 


Number 


Percent 


Number 


Percent 


1 . New England 


70 


6.5 


394 


6.1 


2. Mid East 


197 


18.2 


1,147 


17.9 


3. Great Lakes 


163 


15.1 


945 


14.7 


4. Plains 


85 


7.9 


584 


9.1 


5. Southeast 


223 


20.6 


1,503 


23.4 


6. Southwest 


104 


9.6 


623 


9.7 


7. Rocky Mountains 


40 


3.7 


214 


3.3 


8. Far West 


178 


16.5 


887 


13.8 


9. Outlying Areas 


22 


2.0 


125 


2.0 



'New England includes CT, ME, MA, NH, RI, VT; Mid East includes DE, DC, MD NJ, NY, PA; Great Lakes includes IL, IN, 
MI, OH, WI; Plains includes IA, KS, MN, MO, NE, ND, SD; Southeast includes AL, AR, FL, GA, KY, LA, MS, NC, SC, TN, 
VA, WV; Southwest includes A Z, NM, OK, TX; Rocky Mountains includes CO, ID, MT, UT, WY; Far West includes AK, CA, 
HI, NV, OR, WA; and Outlying Areas includes PR. 

2 Counts obtained from the sampling frame based on the 1998-99 IPEDS IC file. 

NOTE: Details may not sum to 1 00 due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Postsecondary Student Aid Study, 
1999-2000 (NPSAS:2000). 

sent separate lists for each term or course of instruction, in which cases an individual student’s 
name could appear on more than one list. In such cases, the samples were “unduplicated” to 
ensure that each student received only one chance of selection. 18 

As student lists were received from institutions, students were sampled. Stratified 
systematic sampling was used to ensure comparable sampling procedures for both paper-copy 
and electronic lists. In the case of duplicated paper-copy lists, a stratified systematic sample was 
selected from each list provided (typically separate lists by term) and the samples selected were 
“unduplicated” against master lists (see appendix G). 19 After the sample of students had been 
selected for an institution, Social Security numbers (SSNs) of those sampled were compared to 
those of students who had already been selected from other institutions to eliminate cross- 
institution duplication. Multiplicity adjustments in the sample weighting described in more 
detail in Chapter 6 accounted for the fact that any students who attended more than one 
institution during the NPSAS year had more than one chance of selection. 

Initial student sampling rates were calculated for each sample institution using sampling 
rates (see appendix G) designed to generate approximately equal probabilities of selection within 
the ultimate institution-by-student sampling strata. However, these rates were sometimes 
modified for reasons listed below. 



ERIC 



18 Electronic lists were “unduplicated” by sorting on the student identification (ID) number and deleting 
duplicates prior to sample selection. 

19 The baccalaureates were given precedence since a student receiving a bachelor’s degree was sampled as a 
baccalaureate regardless of student type. Next, the fall term was given precedence in this process for comparability 
with NPSAS: 87. If the institution did not have standard terms, other orderings of the student lists were used to 
achieve unduplication of the sample. 
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® The student sampling rates were increased, as needed, so that the sample size 
achieved at each sample institution would be at least 40 sample students, where 
possible. 

• The student sampling rates were decreased if the sample size was more than 50 
greater than the institution had been told to expect, which was based on the sampling 
rate applied to the enrollment count on the sampling frame. 20 

• The sample yield was monitored throughout the months during which student lists 
were received, and the student sampling rates were adjusted periodically for 
institutions for which sample selection had not yet been performed to ensure that the 
desired student sample sizes were achieved. 

These adjustments to the initial sampling rates (especially the first two types of 
adjustments) resulted in some additional variability in the student sampling rates and, hence, in 
some increase in survey design effects (variance inflation — see Chapter 6). 

The planned and achieved sample sizes by student stratum and level of offering are 
shown in table 2-4. The actual sample sizes achieved in total and by school type and student 
stratum are shown in table 2-5. Table 2-4 shows that the overall sample yield was very close to 
what was planned (70,232 students as compared to the target of 70,266). This table also shows 
that overall there were more baccalaureate, master’s, other graduate, and first-professional 
students in the sample than planned, and there were fewer doctoral students than planned. 



Table 2-4. — Planned and achieved NPSAS:2000 student samples, by student stratum 



and level o 


' offering 


Student stratum 1 


Institutional level 2 


Students sampled 


Number expected 3 


Number achieved 4 


Percent 5 


Total 


All institutions 


70,266 


70,232 


100.0 


Baccalaureate business 


4-year 


1,365 


1,475 


108.1 


Baccalaureate other 


4-year 


15,006 


15,147 


100.9 


Other undergraduate 


Subtotal 


40,918 


40,981 


100.2 




Less-than-2-year 


6,925 


6,665 


96.2 




2- to 3-year 


12,653 


13,240 


104.6 




4+ year 


21,340 


21,076 


98.8 


Master’s 


4-year 


5,820 


5,964 


102.5 


Doctor’s 


4-year 


4,543 


3,946 


86.9 


Other graduate 


4- year 


1,293 


1,369 


105.9 


First-professional 


4-year 


1,319 


1,350 


102.4 



*As expected, the sampling frames misclassified some individual students as to baccalaureate, undergraduate, graduate, and first- 
professional status; statistics presented in this table are based on the sampling frame classification. 

Institutional level is based on level confirmed by institution during school contacting. 

3 Based on sample allocation, 1998-99 IPEDS IC file enrollment counts, and 1996-97 IPEDS completions file baccalaureate 
counts. Numbers may not sum to total due to rounding. 

4 The student sample was drawn from 999 institutions determined to be eligible and providing enrollment lists. 

5 Percent reported reflects the ratio of “achieved” to “expected.” 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Postsecondary Student Aid Study 
1999-2000 (NPSAS:2000). 



20 This was to facilitate continued participation by the institutions for CADE data abstraction. 
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Table 2-5. — Initial classification of NPSAS:2000 student sample, by type of institution and student stratum 

Total sample' Student sampling stratum 2 
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2. Design and Method of NPSAS:2000 



2.2.3 Institutional Enlistment and Student List Acquisition and Sampling 

Once institutions were sampled, attempts were made to contact the chief administrator of 
the selected institutions to verify institutional eligibility, solicit participation of eligible 
institutions, and request appointment of an Institutional Coordinator through which subsequent 
communication with the institution would be directed. The initial letter on U.S. Department of 
Education (ED) letterhead included a study fact sheet and endorsement letters, as appropriate for 
that institution, from the National Association of Financial Aid Administrators (NASFAA), the 
American Association of College Registrars and Admissions Officers (AACRAO), the Career 
College Association (CCA), and the National Accrediting Commission of Cosmetology Arts and 
Sciences (NACCAS). Concurrently, NASFAA mailed a separate letter directly to the financial 
aid officers of all member institutions sampled urging participation. (Copies of these letters and 
attachments, as well as other correspondence mailed to sampled institutions or students during 
the course of the full-scale survey are included in appendix B.) Follow-up telephone calls were 
made to the chief administrator one week after the mailing; if the IC had not been named by that 
time, the administrator was urged to name an Institutional Coordinator (with varying degrees of 
success) during the telephone conversation. 

Separate mailings to the Institutional Coordinators (containing all materials included in 
the initial mailing to the chief administrator) were initiated on a flow basis, as the Institutional 
Coordinators were designated. Follow-up telephone calls were, again, initiated one week 
following the mailing (the initial contact with the Institutional Coordinators typically involved a 
series of calls, including refusal conversion calls, since no substitution of refusing institutions 
was employed). Institutional coordinators were advised of what would be expected from the 
institution and asked to verify the IPEDS classification (institutional control and highest level of 
offering) and the calendar system used (including dates that terms started). Institutional 
Coordinators also were asked to (1) provide information on the institution’s record-keeping 
approaches (including identifying the physical on-campus locations of records needed for the 
subsequent record abstraction procedures), (2) identify their PC capabilities for operating the 
CADE software, and (3) set a date by which the school would provide student enrollment lists. 

The list(s) requested (preferably a single “unduplicated” electronic list) were to contain 
all eligible students enrolled in any term within the study-defined year. (Sampled schools with 
additional NPSAS-year terms starting after the date of the request obviously could not provide 
complete lists until after the last applicable term began.) The data items requested for each listed 
student were 

• full name; 

• student identification (ID) number; 

• Social Security number (possibly identical with student ID); 

• educational level — undergraduate, master’s, doctoral, other graduate, or first 
professional — during the last term of enrollment during the study-defined year, 

• for baccalaureate students major field of study for which the baccalaureate degree was 
or will be awarded; and 
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• Classification of Instruction Program (CIP) code for the student’s major. 

Definitions of types of lists and information preferred, as well as instructions for 
preparing different lists, were included in the initial IC letter and further clarified, as needed, in 
follow-up telephone conversations. In such subsequent telephone contacts, contractor staff 
worked closely with the IC to determine the best reasonable alternative lists and student 
information that could be provided by the institution. 

Prompting telephone calls were made to institutions that had not provided lists by one 
week following the most recent delivery date previously agreed upon by the IC. Throughout the 
list acquisition process, attempts were made by the contractor to accommodate school constraints 
and to reduce their burden, including contractor “unduplication” of lists. Where requested, 
institutions were reimbursed for personnel and computer time required to prepare student 
sampling lists. 

Several checks on quality and completeness of student lists were implemented before the 
sample students were selected. Institutions providing lists that failed these checks were called to 
rectify the detected problems. Completeness checks were failed if any of the following 
conditions existed: 

• Baccalaureate recipients/graduating seniors were not identified (unless the institution 
was less-than-4-years or explicitly indicated that no such students existed in the 
school). 

• Student level — undergraduate, master’s, doctoral, other graduate, or first 
professional — was not clearly identified. 

• Major fields of study or CIP codes were not clearly identified for baccalaureates. 

* 

Quality checks were performed by checking the “unduplicated” count from provided lists 
against the “unduplicated” counts from IPEDS and completions files. For applicable institutions, 
separate checks were made for baccalaureate recipients, other undergraduates, graduate, and first- 
professional students; for institutions serving only undergraduates (and no baccalaureates), 
checks were made against total enrollment. The institution failed the check if the count for any 
“unduplicated” list differed by at least 25 percent from the IPEDS count. 21 

2.3 Data Collection and Operational Design 

NPSAS:2000 involved a multistage effort to collect information related to student aid. 

An initial NPSAS:2000 data collection stage collected electronic student aid report (Institutional 
Student Information Report, or ISER) information directly from the U.S. Department of 
Education Central Processing System (CPS) for federal financial aid applications. 22 The second 



21 If provided lists were not “unduplicated,” the contractor estimated the “unduplicated” total by applying an 
empirically determined multiplicity factor (0.50) to the count over provided lists; in these cases, the critical difference also was 
relaxed to at least 30 percent. 

22 The contractor for this service was National Computer Systems (NCS). Students completed a Free Application for 
Federal Student Aid (FAFSA), which was mailed to the CPS contractor; this information was entered into the computer file and 
electronic versions of the Institutional Student Information Record (ISIR) were created. The IS1R information was made 
available to all institutions that the student indicated on the FAFSA. 
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stage involved abstracting information from the student’s records at the school from which he/she 
was sampled, using a computer-assisted data entry (CADE) system. In the third stage, interviews 
were conducted with sampled students, primarily using a computer-assisted telephone 
interviewing (CATI) procedure. Computer-assisted personal interviewing (CAPI) procedures, 
using field interviewers, were also used for the first time on a NPSAS study, to help reduce the 
level of nonresponse to CATI. 

A schematic of the operational flow of major data collection components of the 
NPSAS:2000 study is shown in figure 2-2 and discussed below. To meet established dates for 
conclusion of all activities, while accommodating both differential dates at which student 
sampling could be initiated and differential timeliness of institutional turnaround, not all stages 
were implemented at the same time at all institutions. In fact, the only fixed points in operations 
were (1) selection of the institutional sample plus the initial institutional mailings and 
verification calls, and (2) cutoff of interviewing. Start and end dates for the significant study 
activities were shown earlier in table 1 - 1 . 

2.3. 1 Overview of Data Collection Instruments and Extant Data Sources 

As noted previously, some study data were obtained from extant databases. These 
additional data sources served several useful functions. First, they provided information that 
could not be collected from the institutions or the students. Second, they provided a way to “fill 
in data that was obtained in institutional record abstraction or the student interview but was 
missing for individual sample members (e.g., demographics). Also, additional data sources 
served as a way to check or confirm information obtained from student records or interviews. 

Information related to applications for federal financial aid was obtained (for two 
academic years) from ED's central processing system, the CPS. Additionally, data on the nature 
and amounts of received Pell grant or federal student loans were obtained from the National 
Student Loan Data System (NSLDS) databases maintained by ED. The NSLDS Pell grant and 
loan files that were accessed included information for the 1999-2000 academic year as well as a 
complete federal grant or loan history for each applicable student. In addition to information 
regarding student aid receipt, data were obtained from Educational Testing Service (ETS) for the 
SAT, and from ACT for the ACT assessment, which included test score data as well as additional 
demographic information and some information regarding educational aspirations. 

Obtaining Central Processing System (CPS) information. To reduce institutional 
burden in subsequent data collections, the NPSAS:2000 contractor, with the assistance of NCES, 
arranged to obtain information from the Central Processing System (which was operated for the 
U.S. Department of Education by a separate contractor. National Computer Systems [NCS]), to 
access certain information provided by all federal financial aid applicants who had been selected 
in the sample. Students give this information to the CPS contractor on a Free Application for 
Federal Student Aid (FAFSA) form; it is then converted to electronic form, analyzed, and 
provided to involved schools (and other approved parties). 
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Figure 2-2. — Flow of major data collection components for the NPSAS:2000 study 
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1999-2000 (NPSAS:2000). 



O 

ERIC 






21 



2. Design and Method of NPSAS:2000 



CADE data abstraction from students’ institutional records. Data from sampled 
students’ records at the NPSAS institution were collected using procedures similar to those 
successfully tested and implemented during NPSAS:96. Specifically, a CADE software system 
using version 4.3 of the Computer Assisted Survey Execution System (CASES) 23 , was developed 
for use in collecting data from student records. The data elements included in the Web-CADE 
system (described in more detail in chapter 3) were identical to those included in the laptop- 
based CADE system used by the RTI field data collectors (field-CADE). 

The CADE record abstraction process began when a student sample had been selected 
and transmitted to the Central Processing System for obtaining financial aid application data. 
Upon completion of the CPS matching (typically a 48-hour turnaround), a number of data 
elements were preloaded into the CADE database, thus initializing the CADE system. These 
preloaded elements included an indicator of whether the student had been matched successfully 
to the CPS system, as well as selected CPS variables for use in CADE software edit checks. In 
addition, the system was customized for each institution by preloading of the names of up to 10 
institution financial aid programs and up to 10 state financial aid programs, for use in identifying 
aid received by students. 

As was the case in NPSAS:96, institutions could choose either to enter the data 
themselves or to have an RTI-employed field data collector enter the data. Institutions were 
encouraged to use their own staff for this data collection (with compensation for staff time, when 
requested), since this minimized the overall cost of the data collection. The NPSAS:2000 field 
test demonstrated the effectiveness and user-friendliness of the Web-CADE system, providing 
institutions with further encouragement to complete the data collection themselves. 

Once CADE was initialized for a particular institution, the Institutional Coordinator was 
notified by telephone that the CADE data collection could begin. Coordinators who had 
previously indicated a willingness to complete the data collection via Web-CADE were provided 
with a user name and password to gain access to the Web-CADE systems. As a security 
measure, each coordinator was asked to provide a “lost-password prompting question and 
answer ’ that is, if they forgot their password and had to call in for a reminder, the personalized 
question was posed and the password was provided when they successfully answered the 
question. Field-CADE institutions were also notified by telephone of CADE initialization, at 
which time an appointment was made for a field data collector to visit the institution. 

The CADE software (the full contents of which appear in appendix E) was structured into 
eight sections: 

1 . locating - for collecting address and phone information for students, students' 
parents, and other contacts; 

2. characteristics - for collecting demographic data such as sex, race, and marital 
status; 



23 This software was produced by the Computer-Assisted Survey Methods Program (CSM) of the University 
of California at Berkeley, May 1998. 
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3. admissions - for collecting scores for undergraduate, graduate, and first- 
professional admissions tests; 

4. enrollment - for collecting terms of enrollment, degree program, and field of 
study; 

5. tuition - for collecting tuition data for the terms of enrollment; 

6. financial aid awards — for collecting financial aid data for aid recipients; 

7. need analysis — for collecting student financial aid budget data for aid applicants; 
and 

8. ISIR - for collecting name and SSN for students not previously matched 
successfully to CPS, but for whom an ISIR was available, indicating the student 
had applied for federal financial aid for the study year. 

Because the Web-CADE database was resident on an RTI Web server, daily status reports 
summarizing the progress of the Web-CADE institutions were generated and posted on the 
Integrated Management System (IMS). However, periodic calls were placed to the coordinators 
to inquire as to their progress, thereby prompting the institutions to complete the record 
abstraction. In general, status reports indicated that schools were typically slow in beginning the 
CADE task (often waiting many weeks after system initialization before starting data collection), 
but once they began they tended to complete the task relatively quickly. 

Student CATI/CAPI interviews. Student interviews were conducted primarily by 
telephone, and occasionally in person, using CATI/CAPI technology. Like CADE, CATI/CAPI 
was developed using version 4.3 of the Computer- Assisted Survey Execution System (CASES) 
software to facilitate preloading full-screen data entry and editing of “matrix-type” questions. 

The CATI/CAPI system presented interviewers with screens of questions to be asked of the 
respondents, with the software guiding the interviewer and respondent through the interview, 
automatically skipping inapplicable questions based on prior response patterns or suggesting 
appropriate wording for probes should a respondent pause or seem uncertain in answering a 
question. 

To reduce interview burden and to guide the interview through appropriate branchings 
(e.g., questions appropriate only for graduate students), considerable information was preloaded 
into the CATI records before the interviews. Such preloaded information included (1) data 
previously collected through CPS and/or CADE; and (2) information from the sampling file (e.g., 
name, Social Security number, school name, school and student stratum). In a number of 
instances, specific questionnaire items were not asked (or were only verified) if that information 
had been collected previously. Data were preloaded into CATI on a flow basis, as CADE results 
were received from the institutions. 

Features of the CATI system that facilitated smooth and appropriate conduct of the 
interview included: 

• extensive use of appropriate branching of interviewees based on preloaded 
information or responses to questions asked previously in the interview; 
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® extensive use of “fill” features in screen presentations of questions to be asked by 
interviewers (i.e., filling in part of a question with preloaded data or a previously 
provided response — that is, instead of asking the respondent something about “second 
postsecondary institution that they attended,” the question would be presented with 
the name of the institution embedded in the screen wording); 

• a “breakoff/resume” feature allowing interview continuation after a breakoff to move 
automatically to the next applicable question for the respondent; and 

® provision of context-sensitive “help” screens (available with a single keyboard entry) 
to provide the interviewer with information about particular questions to help clarify 
the question's intent. 

Additionally, online coding programs developed by NCES (for industry/occupation, 
IPEDS, and field of study coding) were embedded in the overall interview administration system. 
These allowed standard coding of verbatim responses while the respondent was still available to 
assist. 



The student CATI interview consisted of seven sections that were administered 
sequentially (see figure 2-3). 24 The sections were ordered so that important information was 
collected early in case the respondent broke off the interview before completion. A facsimile 
student interview is provided in appendix D. 

Cases not completed in CATI (i.e., refusing and/or unlocatable cases) were assessed for 
assignment to field staff. If the case was in an identified geographic cluster, it was assigned to a 
field interviewer. The field interviewer then attempted to locate the student and complete the 
interview using CAPI. If the case was not in an identified cluster, it was assigned to a field 
locator. The field locator then attempted to locate the student and convince the student to call an 
800 number to complete the interview in CATI. 

Results of CATI and CAPI interviewing were monitored daily through the study 
Integrated Management System (IMS). Daily reports of production, with revised projections of 
future production to satisfy study requirements, were available to both NCES and contractor 
staff. 



Two sets of abbreviated interviews were conducted in special cases. First, the planned 
reliability reinterview study used an interview containing only a small subset of the items in the 
full student interview. Second, an abbreviated interview was developed in English and Spanish 
(containing only selected items) for telephone administration to those who were Spanish- 
speaking only 25 sample members or for use in refusal conversion. Facsimiles of the reliability 
interview and the abbreviated interview are provided in appendix F. 



While the logical flow within an interview is generally constrained to be linear (with forward branching 
as applicable), this is even more important in CATI, where previously supplied responses control subsequent 
branching items. Nonetheless, standard features were available to allow interviewers to back up in the interview to 
change prior responses based on information provided subsequently. 

25 Spanish speakers who could speak some English were guided through the full interview by bilingual 
interviewers. However, translation “on the fly” of the full interview to one who spoke only Spanish was considered 
inappropriate, and thus the Spanish translation of the abbreviated interview was administered in these cases. 
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Figure 2-3. — Structure and flow of NPSAS:2000 student CATI 




SOURCE: U.S. Department of Education, National Center for Education Statistics, National Postsecondary Student Aid Study, 
1999-2000 (NPSAS:2000). 
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Other /ws/ hoc student record data obtained. The electronic data interchange with the 
National Student Loan Data System (NSLDS), (including both loan and Pell grant files), ACT 
database, and ETS SAT files was initiated toward the end of CATI operations. As with the 
previously described procedures with CPS, matching of students to these files required Social 
Security numbers. At the time of these requests, apparently valid SSNs were available for 69,449 
sample members, the number subsequently submitted for all attempted matches and associated 
data downloads. 26 In addition to SSNs, name and date of birth were submitted to ETS for SAT 
matching and to NSLDS for Loan and Pell matching. For ACT, sex and date first enrolled (if 
available) were included in the file along with name and date of birth. These variables assisted 
the data vendors in performing confirmatory data quality checks. All matching processes were 
initiated by RTI staff providing a file with one record per sample member with the requested data 
on a CD-ROM to the database system. A successful match with the NSLDS loan and Pell 
database required that the student have a valid application record within the database. Similarly, 
a successful match with the ACT and SAT databases required that the student have a valid record 
with the test databases. Additional data (e.g., date of birth) was used when necessary to increase 
the likelihood of a successful and accurate match. 

2.3.2 Student Locating 

The basic NPSAS:2000 design involved tracing sample members to their current location 
prior to conducting a computer-assisted telephone interview or a computer-assisted personal 
interview with them. 

Pre-CATI locating. Locating information obtained during the institutional CADE phase 
of the study was incorporated into the locator database. The data files were updated in batch 
mode to the National Change of Address (NCOA) 27 system and Telematch 28 on a flow basis. 

After the locator database had been updated with the new information, a lead letter packet was 
mailed to the best address for the sample member. This mailing included a standard lead letter 
and a study leaflet. These mailings occurred on a flow basis twice a week beginning in May 
2000 and continued throughout the data collection period. The most current information for the 
student and any other contacts were then preloaded into the CATI system to assist the 
interviewers in locating the sample members. 

CATI-internal locating. When assigned a case, the telephone interviewer called the 
telephone number designated by the system as the best number (i.e., the number among all 
available locator numbers that appeared to have the greatest potential for contacting the sample 
member) and attempted to interview the designated sample member. When the person answering 
the call said that the sample member could not be reached at that number, the interviewer asked 
the person how to contact the sample member. If this query did not provide the information 

26 Of these, 8,120 were ultimately determined to be nonrespondents. 

27 NCOA is a database consisting of change of address data submitted to the U.S. Postal Sendee. Almost 
100 million records are updated every 2 weeks and stored for 3 years. 

Telematch is a computerized residential telephone number look-up service consisting of over 65 million 
listings, over one million not-yet-published numbers of new movers, and over 10 million businesses. Telematch uses 
a name, street address, and ZIP code as search criteria and Reverse Telematch uses telephone numbers as the search 
criteria to provide the names under which telephones are listed. 
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needed, the interviewer initiated tracing procedures, using all information available to call other 
contact persons in an attempt to locate the sample member. When all tracing options available to 
the interviewer were exhausted without success, the case was assigned to intensive tracing via 
FastData 29 , Tracing Operations Unit (TOPS) 30 , or field interviewers/locators. The latter two 
intensive tracing steps are described below. 

Intensive locating (post-CATI tracing). All cases that were not located during the 
CATI locating process were submitted to TOPS for intensive locating. TOPS implemented a 
two-tiered intensive tracing plan. The first tier involved identifying sample members with Social 
Security numbers and processing that information through a series of electronic databases. The 
specific tracing activities are listed below, and were restricted to the collection of 
locating/directory information. 

• Query > of Equifax database. Equifax is a credit bureau that maintains credit files on a 
large number of individuals. 

• Query of Internet databases. Contractor staff had direct electronic access to various 
databases, which included names, Social Security numbers, and current and former 
addresses and telephone numbers of individuals. 

• Query of the Select Phone Book CD ROM data. This database contains every 
published telephone number in the United States, with associated names and 
addresses. It can be sorted within city by address, to obtain telephone numbers and 
names of neighbors. 

If the searches generated a new telephone number, that case was sent back to RTFs 
Telephone Survey Department (TSD) for telephone interviewing. If a new address was 
generated, but no telephone number, tracers called Directory Assistance or accessed other 
databases to obtain telephone numbers for the TSD. This first level of effort minimized the time 
that cases were out of production. 

All remaining cases (those lacking new information from the SSN search) underwent a 
more intensive level of tracing in the second-tier approach. This approach involved the 
following procedures: (1) checking Directory Assistance for telephone listings at various 
addresses; (2) using electronic reverse-match databases to obtain the names and telephone 
numbers of neighbors and then calling the neighbors; (3) calling persons with the same unusual 
surname in small towns or rural areas to see if they were related to or knew the sample member; 
(4) contacting the current or last-known residential sources such as neighbors, landlords, current 
residents, tax assessors, realtors, and other business establishments related to previous addresses 
associated with the sample member; (5) calling colleges, military establishments, and 



29 FastData is a series of database searches used to locate sample members after pre-CATI batch database 
searches have been done but before sending cases for intensive interactive tracing. 

30 The Tracing Operations Unit (TOPS) is a highly specialized unit within RTI that was created in response 
to the recurring needs of certain research methodologies to locate large numbers of sample members. The sole focus 
of TOPS is tracing sample members so that they can be located for research studies; the unit does not involve any 
data collections. 
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correctional facilities to follow up on leads generated from other sources; and (6) checking 
various tracing Web sites. Tracers checked new leads produced by these tracing steps to confirm 
the address and telephone numbers for the sample members. When the information was 
confirmed, the case was returned to CATI for completion. If the information could not be 
confirmed (e.g., there were no working telephone numbers or numbers for relevant neighborhood 
sources were unpublished), the case was sent to the field. 

Field locating. The main purpose of the intensive field locating/interviewing effort was 
to increase the response rate. However, since the costs of conducting these operations were high, 
field efforts were implemented only when less costly efforts were exhausted. Sample members 
were identified as needing field locating/interviewing if they were not located using CATI- 
locating and centralized intensive tracing. 

Geographic clusters of sample members were designated, and 33 of these clusters were 
staffed with field interviewers who were trained to locate sample members and interview them 
using a laptop computer Field cases falling outside the geographic clusters were assigned to field 
locators (trained as interviewers on other RTI studies) who located sample members in their local 
areas and encouraged them to call in to RTFs TSD to be interviewed. 

2.3.3 Telephone Interviewing 

CATI locating and interviewing began on May 22, 2000, and continued through 
February 28, 2001 . CATI procedures included attempts to locate, gain cooperation from, and 
interview study sample members by telephone. 

Before the CATI sequence began, notification letters on U.S. Department of Education 
stationery and with attachments were mailed to students. These letters notified the sample 
members of the upcoming survey, pointed out the importance of the study, disclosed average 
time burden, and urged participation. 

Associated with the interviewing was the necessity (due to incomplete or incorrect 
telephone numbers), in many cases, to locate the respondent(s). Much of the locating challenge 
was associated with the fact that many NPSAS:2000 sample members (particularly those who 
had just received their degrees) were at a stage in their lives in which they were highly mobile. 

To facilitate the tracing component, each CATI record contained roster lines for up to 15 
telephone numbers; each such roster line was associated with a history of the dates and results of 
all calls made to that number and a number-specific comment field. Up to five roster lines were 
preloaded with contact information. New roster lines were added during CATI tracing operations 
as a result of locating sample members via intensive tracing efforts. Locating calls were 
initiated according to a calling plan using an automatic call scheduler embedded within the CATI 
software. This system allowed calls to be scheduled on the basis of established case priority, 
time of day, and history of success of prior calls at different times and on different days. 
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Once located, an attempt was made to conduct the full interview with the sample 
member. However, some cases required special treatment. To deal with those who initially 
refused to participate (including locator sources who acted as “gatekeepers,” preventing access to 
the sample member), certain interviewers were trained in refusal conversion techniques. Sample 
members and their locator sources who spoke only Spanish, primarily located in Puerto Rico, 
were assigned to bilingual CATI interviewers. 

Finally, in an effort to increase study response rates, a modest incentive was used with 
particular types of nonrespondents: (1) cases where the sample member initially refused the 
interview; (2) sample members for whom intensive tracing yielded a good mailing address, but 
no telephone number; and (3) cases identified as “hard to reach” (i.e., those with 20 or more call 
attempts, where contact had been established with the sample member and no “hard” 
appointment was pending). The incentive consisted of a letter from the project director on RTI 
letterhead, tailored to the specific type of nonrespondent (i.e,, refusal or hard to reach/no 
telephone number). A $5 bill was included with the letter. Respondents were promised a check 
for $15 if they called an 800 number to complete the interview. The incentive letters were mailed 
on a flow basis as respondents met one of the three criteria described above. All cases sent to 
field interviewers or field locators were automatically made eligible to receive the incentive once 
the case was sent to the field. Interviews were obtained from about half of the sample members 
who were offered the incentive with almost 60 percent of those initially refusing being converted 
by the incentive offer. 

2.3.4 Field Interviewing 

Field interviewing activities began after training was conducted and field cases and bulk 
supplies were shipped to the field interviewers. CAPI procedures included attempts to locate, 
gain cooperation from, and interview study sample members either by telephone or in person. 

All students who were finalized in CATI and by TOPS as “unlocatable” were eligible for 
assignment to the field for CAPI interviewing or field locating. Sample members who had not 
completed the NPSAS:2000 interview at the time field interviewing began and who resided in an 
identified geographic cluster in the vicinity of a field interviewer were immediately assigned to 
the field. The field interviewer then attempted to locate the student and complete the interview 
using CAPI. If the case was not in an identified cluster, it was assigned to a field locator. The 
field locator then attempted to locate the student and convince the student to call an 800 
telephone number to complete the interview in CATI. 

Field interviewers documented every telephone call or field contact. They were provided 
with a checklist that included example questions to help with tracing operations and that 
demonstrated the correct order in which tracing activities should be performed. The checklist 
was completed for each case to help identify the sources that were most useful in locating the 
students. 

Primary tracing sources included parents, current or former neighbors or roommates, the 
NPSAS school, and city and county offices. Secondary tracing sources included Directory 
Assistance, the Chamber of Commerce, public libraries, the U.S. Postal Service, and the 
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Department of Motor Vehicles. Other miscellaneous sources for field interviewers, useful in 
some cases, included small town police or sheriffs departments, fire departments or emergency 
rescue squads, local newspapers, public housing authorities, mobile home park managers, motel 
staff, probation officers, and permit-issuing departments at the city level (new construction). A 
contact script guided interviewers in soliciting information from various sources. 

When field interviewers successfully located sample members, they introduced them- 
selves and explained the purpose of the study, referring to the advance letter mailed previously. 
They then attempted to complete the interview using the same instrument used in the CATI 
interview. The field staff were supported by a computerized control system that tracked field 
assignments and captured pending and final result codes. Daily reports, posted to the IMS, 
tracked the progress of the field effort. 

2.3.5 Training CADE Data Collectors 

The training for RTI CADE staff was held in two sessions to allow for efficient use of the 
field staff immediately following training. Prior to these sessions, six Field Supervisors hired for 
the CADE collection were trained in February 2000. The initial training for 23 CADE Field Data 
Collectors was conducted during April 2000. The second session was originally planned for 
June; however, this session was postponed to late July 2000 to coincide with the projections of 
list receipt from institutions, sample selection, and flow of cases into CADE. Staff scheduled to 
attend the June session were notified of the delay and there were no attrition problems related to 
the postponement. Five of the six Field Supervisors attended and participated in the training 
session and 13 Field Data Collectors successfully completed the session. To reduce travel costs 
for the relatively small number of trainees, the training sessions were held in the Research 
Triangle Park area. 

The Field Supervisor training included a 2-day session on the background of 
NPSAS:2000 (including objectives, time frame, and the financial aid process), supervisory and 
administrative responsibilities, procedures for recruiting field data collectors, and use of the Case 
Management System, the assignment and transfer (WebATS) system, and the e-mail system. 

The Field Data Collector training included a half-day of training on the computer for a subset of 
the trainees (who needed an introduction to the computer) prior to the project training. Training 
consisted of an overview of the NPSAS:2000 objectives and time frame, explanation of how the 
financial aid process works on campuses, review of the architecture and nature of the CADE 
software, review of and practice with each section of the CADE instrument, procedures for 
contacting and dealing with the Institutional Coordinator and other staff at the institutions, 
instruction in and practice with locating records (including, but not restricted to use of the 
“location of records” lists provided by the Institutional Coordinators and review of ISIRs, 
procedures for contacting Field Supervisors, electronic transmission of completed cases, and 
administrative procedures. 

During this training, considerable use was made of location and abstraction of records 
using mock student folders developed, with the assistance of NASFAA staff, to represent 
diversity in record keeping at different types of postsecondary institutions. Laptop computers 
were provided to all trainees for their use during training and subsequent field work. 
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Additionally, as a training aid, each trainee was issued a Field Data Collector Manual 31 and a 
CADE Users ’ Guide. 32 The tables of contents for both of these manuals as well as a copy of the 
Field Data Collector training agenda are included in appendix D. 

Training of institutional staff in use of the Web-CADE application relied heavily on self- 
training, since the major objectives of that training were to become familiar with the CADE 
program and to learn how to access the program through the World Wide Web. A secure user ID 
and password were required to access the system. Help screens were embedded within the 
program and a “hotline” number and e-mail address were established through which users could 
obtain answers to specific or general questions from RTI central office staff who developed the 
software. Additionally, institutional staff were provided with a copy of the CADE Users ’ Guide. 

2.3.6 Training of CATI/CAPI Interviewers and Tracing Specialists 

The mixed-mode design of the NPSAS:2000 student data collection required the 
development of three separate training programs: CATI interviewing, field interviewing, and 
tracing. Each training program consisted of separate protocols for data collectors and for 
supervisors. For each, training topics covered administrative procedures, including 
confidentiality requirements and quality control techniques; student locating; interactions with 
students, parents, and other contacts; the nature of the data to be collected; and the organization 
and operation of the CATI, CAPI, and tracing operations systems used for data collection. The 
goals for these training programs were to 

o increase the accuracy, quality, and timeliness of the data collected; 

® standardize the quality of data collection techniques and procedures; and 

® provide explicit, nonjudgmental procedures for telephone interviewers, telephone 
monitors, field staff, tracing specialists, and supervisors to follow. 

Training telephone interviewers. Initial training for telephone interviewers, monitors, 
and supervisors began in late April 2000 immediately before student data collection started. 

Most of the supervisors and monitors used on the project were trained in a separate session, prior 
to interviewer training, so that they could assist during subsequent training sessions. Because 
cases flowed into CATI over time from the school data collection effort (rather than being loaded 
all at once at the outset of data collection), it was necessary to schedule the required training 
sessions over time to mirror the CATI workload. In all, 23 training sessions were held for CATI 
interviewers, monitors, and supervisors between April and December 2000. In total, 372 
telephone interviewers were trained over this 9-month period. Table 2-6 lists the training sessions 
offered and the number of interviewers, supervisors, and monitors completing each training 
program. 



31 RTI Field Data Collector Manual: NPSAS.2000 Main Study. Research Triangle Park, NC, March 2000. 

32 RTI CADE Users ’ Guide: NPSAS:2000 Main Study. Research Triangle Park, NC, April 2000. 
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Table 2-6. — CATI training activities and number of interviewers trained 



Training activity 


Number of 
sessions 


Number of people 
trained 


CATI supervisor/team leader training 


i 


37 


CATI monitor training 


i 


20 


General telephone interviewer training 


16 


297 


Telephone interviewer training and refusal avoidance 


20 


372 


Telephone interviewer refusal conversion training 


8 


86 


Tracing specialist training 


9 


106 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Postsecondary Student Aid Study, 
1 999-2000 (NPSAS:2000). 



Newly hired interviewers with no prior telephone interviewing experience were also 
provided with 8 hours of general or introductory CATI training before they were allowed to 
attend the project specific training. In these sessions, new interviewers were instructed on 
general interviewing techniques and best practices, the screen layout and coding conventions 
used on all CATI projects conducted at RTI, and the routine administrative procedures and 
requirements for working in RTFs Telephone Survey Department. New interviewers who did 
not successfully complete the 8 hours of general training were not allowed to proceed to the 
project-specific NPSAS:2000 training. 

Project-specific training for C ATI-experienced telephone interviewers and new hires who 
successfully completed general interviewer training consisted of 20 hours of classroom and 
practical, hands-on training. Topics covered included the nature and purpose of NPSAS:2000 
and the B&B:2000/2001 follow-up; the procedures and protocols to be used for tracing, 
contacting, and interviewing sample members; and an extensive review of the NPSAS:2000 
instrument. During the training, all questions in the interview were reviewed, and interviewers 
received both written and hands-on practice with the screens and subroutines for conducting 
online coding, and time for both group and individual practice with the instrument itself. 
Prescripted or “mock” interviews were designed to ensure that interviewers received hands-on 
practice with the most common paths through the questionnaire as well as practice administering 
some of the more difficult items in the questionnaire. Small group training, using audiotaped 
scenarios, was also provided to enhance refusal avoidance skills. At the end of the project- 
specific training, all interviewers were required to complete a certification process to ensure their 
readiness to conduct efficient and reliable interviews for the project. The certification process 
involved the successful administration of the NPSAS interview in a paired “mock” situation with 
a fellow trainee (one playing the interviewer, the other the sample member). Trainers monitored 
these sessions, noting any difficulties a trainee might have had with questionnaire administration; 
use of online coding programs; keying accuracy; and voice tone, speed, and quality. Those who 
did not successfully complete the training and pass the certification process were not allowed to 
work on the study. 
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At the outset of the training, each interviewer received a detailed NPSAS:2000 Telephone 
Interviewer Manual 33 that served as both an instruction guide for the training’s lectures, 
discussions, and practical exercises, and as a reference guide for use after completion of training. 
The manual’s table of contents and a sample of the training agenda for telephone interviewer 
training are included in appendix D. The interviewer manual, supplemented with additional 
materials more directly related to supervisory activities, was also provided to telephone 
supervisors and monitors. 34 The supplementary materials included data collection schedules and 
staff contact information, procedures for supervising interviewers during data collection, tracing 
review and other quality control activities, problem resolution, refusal avoidance and conversion 
techniques, and administrative and record-keeping activities. 

Staff involved with interviewer monitoring received 2 hours of additional instruction on 
the protocols and procedures for conducting interviewer performance monitoring and quality 
assurance monitoring. The training included a review of the interviewer performance monitoring 
form and hands-on practice with the online program developed for quality assurance monitoring. 
Each monitor received a separate manual documenting the procedures to be followed. 35 

Six weeks after the start of student interviewing, project staff began conducting a series of 
refusal conversion trainings for a subset of the highest-performing telephone interviewers. CATI 
supervisors and monitors evaluated the effectiveness of telephone interviewers in dealing with 
respondent objections and overcoming barriers to participation. The most effective interviewers 
received additional and specialized instruction in specific refusal conversion techniques, 
including obtaining cooperation from sample members, addressing concerns raised by parents 
and other sample gatekeepers, validating the importance of the study, and encouraging 
participation among sample members who were nonrespondents in the previous data collection. 
During the course of data collection, 86 interviewers completed refusal conversion training. 

Training field interviewers. To ensure standardization and reliability in the field data 
collection effort, all field interviewing and supervisory staff were required to complete a 32-hour 
comprehensive training program designed to maximize both data quality and interview response 
rates. This training program included classroom lectures, hands-on practice, and other practical 
exercises. The content of the training sessions focused on an overview of the nature and purpose 
of NPSAS:2000 and the B&B:2000/2001 follow-up, procedures for tracing and contacting 
sample members in the field, an extensive question-by-question review of the NPSAS:2000 
instrument, practice with the interview screens and online coding programs, and time for both 
group and individual practice. 

As with the telephone interviewer training, the field interviewer training program 
provided hands-on training with the CAPI interview program. Additionally, the training program 
covered tracing techniques, contacting protocols, and case management, including the use of 
electronic mail and data transmissions systems, troubleshooting guidelines for the laptop 
computer, and field-specific reporting and administrative requirements. 



33 Telephone interviewer Manual for NPSAS.2000. Research Triangle Park, NC: April 2000. 

34 RTI Telephone Supervisor’s Manual for NPSAS.2000. Research Triangle Park, NC: April 2000. 

35 RTI Monitor Manual for NPSAS.2000. Research Triangle Park, NC: April 2000. 
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Each interviewer received a copy of the NPSAS:2000 Field Interviewer Manual 36 at the 
start of the training. This manual, which served as both an instructional resource and a reference 
book for the field work, introduced and reviewed many topics important to the study. The 
classroom instruction, discussion, and practical exercises focused on general interviewing, field 
tracing, and student contacting. The manual and field interviewer training also provided 
instruction for reviewing the case history documentation generated by in-house tracing activities 
to avoid repeating steps taken during earlier tracing efforts (e.g., telephone interviewer contacts 
and centralized tracing efforts). The interviewer manual, supplemented with additional materials 
more directly related to supervisory activities, was provided to field supervisors. 37 The 
supplementary materials included data collection schedules and staff contact information, 
procedures for supervising interviewers during data collection, tracing review and other quality 
control activities, problem resolution, interview verification procedures, and administrative and 
record-keeping activities. 

Initial training for field supervisors took place in August 2000, several weeks before the 
first field interviewer training session. These supervisors then assisted with the initial training 
for field interviewers that took place in September, just before the start of field data collection. 
Two more training sessions were held for additional field interviewers in November and 
December. Overall, 6 field supervisors and 74 interviewers completed the field interviewer 
training for NPSAS:2000. 

Finally, 65 field locators, who were used to assist with tracing of unclustered 
nonrespondent cases, were trained using a home-study packet, rather than a centralized training 
program. As case assignments were made, each field locator was sent home-study materials 
consisting of a study overview, a field locator manual that explained the nature of the assignment 
and the steps to be followed in locating hard-to-find sample members, instructions for making 
contact with sample members and other potential contacts, and a set of example tracing 
materials. Field locator assignments were made initially in October 2000 and continued through 
January 2001 . 

Training tracing specialists. Staff working in RTFs TOPS on the centralized locating 
and tracing activities for NPSAS:2000 also received project-specific instruction, although not as 
extensive as the programs developed for telephone and field interviewers. Each tracing specialist 
received two hours of instruction, including an overview of the nature and purpose of 
NPSAS:2000 and the B&B:2000/2001 follow-up; the study schedule; protocols for contacting 
sample members, gatekeepers, and other contacts; the tracing steps and techniques to be used for 
locating NPSAS:2000 sample members; and the tracing-specific reporting and administrative 
requirements for the study. 

Newly hired tracing specialists also received 8 hours of general tracing instruction. This 
training focused on general tracing techniques; use of the computer search resources in TOPS; 
documentation of locating steps in the TOPS case management system; techniques for obtaining 
locating information for sample members from parents, gatekeepers, and other contacts; and the 



36 RTI Field Interviewer Manual for NPSAS:2000. Research Triangle Park, NC, May 2000. 

37 RTI Field Supervisor Manual for NPSAS:2000. RTP, NC, May 2000. 
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general and routine procedures for working in the TOPS unit. Tracers who did not successfully 
complete the general tracing training were not permitted to attend the project-specific training. 

Eight training sessions were held between May and November 2000 for tracing staff. In 
total, 8 tracing supervisors and 83 tracing specialists were trained to work on NPSAS:2000. 

2.3.7 Evaluation and Quality Control Design 

Each major component of the NPSAS:2000 full-scale study was evaluated. Formative 
evaluations were designed to assess tasks at intermediate stages so that the effects of employing 
alternate methodologies could be analyzed, and modifications and revisions could be employed 
and assessed prior to task completion. Other evaluations assessed the ultimate outcomes of the 
survey. A summary of NPSAS:2000 evaluations that were planned and implemented is provided 
in table 2-7. 

As indicated in table 2-7, the study design included a number of components for 
evaluation of data quality. Among these, a reliability reinterview was conducted with students 
about 8-12 weeks after the initial interview; this involved a random subsample of respondents to 
the initial interview. The reliability reinterview contained only a small subset of the initial 
interview items. Also critical to the operational evaluation and quality control were the regular 
quality circle meetings with field interviewers, telephone interviewers, interview monitors, and 
interviewer supervisors. These meetings provided an easily available forum for production staff 
and project management to address the important topic of work quality, discuss issues of 
concern, identify problems with the survey instruments, share ideas for improving the 
instruments, and suggest various approaches for improving operations and/or results. To 
implement suggested improvements arising from these meetings, the operational features of the 
CATI instrument were sometimes refined over the course of the data collection period. On 
completion of data collection, final quality circle meetings were held, serving as debriefing 
sessions for the full operational period. 

2.4 The Integrated Management System 

The NPSAS:2000 IMS was developed based on a framework initially developed (and 
refined) under previous NCES studies conducted by RTI. These include BPS:90/92, BPS:90/94, 
NPSAS:96, and BPS:96/98. As with these previous studies, the NPSAS:2000 IMS consisted of 
independent, but integrated, modules. Development of the IMS occurred throughout the study 
field test period, and was modified before the full-scale study based on field test results. To the 
extent possible, the NPSAS:2000 IMS .was developed using commercial, off-the-shelf PC-based 
software systems. 
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Table 2-7. — Summary of NPSAS:2000 evaluations 



Major area of evaluation 



Evaluation approaches 



Training 



Enrollment file acquisition 



Record abstraction 



Tracing activities 



Interview administration/data quality 



Debrief field abstractors.* 

Debrief CATI staff.* 

Analyze overall response rate, accuracy, costs, and time to produce lists. 

Evaluate electronic file matching/downloading approaches. 

Analyze data quality (missing data) under conditions of web-CADE, 
field-CADE, and data file production approaches. 

Debrief institutional coordinators.* 

Debrief field staff.* 

Debrief tracing staff and supervisors.* 

Analyze all levels of tracing results and costs. 

Analyze silent monitoring quality control data. 

Analyze CATI operational parameters (e.g., numbers of calls per case,* 
total interviewer hours per completed interview). 

Analyze interview response burden, overall and by section. 

Debrief interviewers, monitors, and supervisors.* 

Analyze response rates and patterns of interview nonresponse. 

Analyze impact of financial incentive on response rate. 

Analyze response temporal stability (reliability) through reinterviews of 
selected items. 

Analyze effectiveness of various strategies for handling answering 
machines. 



I 



^Informal debriefings of staff involved in different data collection tasks were conducted throughout the study. Information 
gathered through these debriefings was used to enhance understanding of the outcomes of more formal evaluations and is 
therefore not described separately in this report. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Postsecondary Student Aid Study, 
1999-2000 (NPSAS:2000). 



The major enhancement to the NPSAS:2000 IMS was the development of a Web-CADE 
module for institutions to provide student data via the Internet.. The system replaced the diskette- 
based version of CADE used during NPSAS:96. The Web-CADE system included encrypted 
data transmission and a login/logout feature to maintain data security. More information about 
Web-CADE is provided below. 

The modular design of the IMS allowed for efficient upgrading or replacement of 
components, or modules, as necessary. This occurred during the field test period, as RTFs 
migration from SQL Server 6.5 to SQL Server 7.0 took place during the summer of 1999. 

Below are listed the major modules of the NPSAS:2000 IMS. Relevant details regarding 
each module are provided. 
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Receipt Control System (RCS) 

® Back-end database is Microsoft SQL Server. SQL Server version 6.5 was used for the 
field test development. The RCS back-end database was upgraded to SQL Server 
version 7.0 near the end of the field test period and before the full-scale study. 

® Front-end interface was programmed in Microsoft Visual Basic 5.0 and Microsoft 
Access 97. 

® RCS reports were developed using Crystal Reports 6.0 and Microsoft Access 97. 

Web CADE 

® Back-end database was Microsoft SQL Server 6.5 (subsequently upgraded to version 
7.0). 

• Front-end interface was programmed in HTML. 

• Middleware software, which allows the Web pages to communicate with the back-end 
database, was Allaire Cold Fusion version 4.0. 

• Web-CADE edit checks were programmed using JavaScript. 

• Reports were developed using Crystal Reports 6.0, Microsoft Access 97, and Cold 
Fusion 4.0. 

® Web security was implemented using Secure Socket Layer (SSL) certification with 
128-bit encryption. User IDs and passwords were assigned by RTI using Microsoft 
Windows NT 4.0 domain security. 

• Users' browsers were required to support, and be enabled for, JavaScript. 

Field CADE 

• The field CADE system was run on Toshiba Satellite laptop computers configured 
with 32MB of RAM and Pentium processors. 

• Back-end database was CASES version 4.3. 

• Instrument was programmed in CASES 4.3. 

® User exits were programmed using C++. 

® Final CADE database was maintained in SAS version 6.12. 

• CADE quality control reports and status reports were programmed in SAS 6. 12. 
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CATI/CAPI 

© Back-end database was CASES version 4.3. 

© Main instrument was programmed in CASES 4.3. 

• Abbreviated instrument (for use in refusal conversion and hardcopy format) was 
programmed in CASES 4.3. 

• CATI user exits were programmed using C++. 

• Final CATI database was maintained in SAS 6.12 (subsequently upgraded to SAS 

8 . 1 ). 

• CATI status and summary reports were programmed in SAS 6.12 (subsequently 
upgraded to SAS 8.1). 

• The CATI system was ported to a CAPI version, for use in conducting in-person 
interviews with students. The same software systems were used for the CAPI system, 
with the exception of a case management component developed in SQL Server 7.0 
and Visual Basic 5.0. 

Data Library 

• CD-ROM-based searchable database of Data Library entries was maintained in SQL 
Server 7.0 throughout the course of the study. The Data Library was initialized 
during the NPSAS:2000 field test. 

• Web-based searchable database of Data Library entries was programmed in Cold 
Fusion 4.0 and Microsoft Access 97. 

• Word processed documents were created using Microsoft Word. 

• Spreadsheets were created in Microsoft Excel. 

• Schedule files were maintained in Microsoft Project 98. 




C o 



38 



2. Design and Method of NPSAS:2000 



IMS Web site 

• Infrastructure was programmed in HTML, with Cold Fusion 4.0 providing “action 
pages.” 

• SQL Server 7.0 served as the back-end database where applicable (maintaining the 
project staff contact list, Technical Review Panel membership, confidentiality report, 
etc.) 

Central Processing System (CPS) 

• Back-end database for CPS data received was SAS version 6.12 and version 8.1. 

• The CPS was a mainframe-based system called the Title IV Wide Area Network 
(T4WAN). Communications with T4WAN were through EDConnect for Windows 
version 2.3. 

• CPS input files were prepared using SAS 6. 12 / 8. 1 . Input files were flat ASCII files, 
with the Federal Data Request (FDR)-file layout (as specified in the CPS Electronic 
Data Exchange Technical Reference manual). 

• CPS data files were read using SAS 6.12/8.1. CPS data files were flat ASCII files 
(one record per student, plus header and trailer records) with FDR full ISIR layout (as 
specified in the CPS Electronic Data Exchange Technical Reference manual). 

National Student Loan Data System (NSLDS) processing 

• Input files for matching to the NSLDS were created as flat ASCII files, containing 
student name, SSN, and date of birth. Files contained one record per sample student. 

• NSLDS data were received as ASCII files containing loan-level transactions (multiple 
records per student). NSLDS loan records reflected cumulative history of loan data 
(i.e., not just the NPSAS year). 

• Pell Grant data files were also received from NSLDS as flat ASCII files containing 
Pell-award-level records. As with the above-mentioned loan data, each student’s 
cumulative Pell history was obtained. 

• All NSLDS input files were created and processed using SAS 6. 12 / 8. 1 . 

® Back-end database for all NSLDS data was SAS 8.1 format. 
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Admissions test file processing 

® Student SAT data (scores and background variables) were obtained from ETS. ACT 
scores and background variables were obtained from ACT. 

® Input files for submission to ETS and ACT were flat ASCII files, containing student 
name, SSN, and date of birth. Files contained one record per sample student. 

® Admissions test files (received back from ETS and ACT) were flat ASCII files 
containing student-level records (one record per student). A separate file was 
received for each admissions test cohort year (multiple files received from each 
admissions test vendor). 

® Input files for admissions test data were created and processed using SAS 6.12/8.1. 

• Back-end database for admissions test data was SAS 6.12. 

Automated processing 

During full-scale data collection, a series of automated batch files were executed nightly 
via Windows NT scheduled processing. These automated processes included the following. 

o Zero record update 

Each night a process would run to copy the CATI “Zero” record (i.e., the master case 
status file) to an SQL table within the RCS database. This information was used to 
synchronize files between the RTI call center and the data being collected by field 
data collectors. The two key synchronization fields were the current status (interview 
complete, pending, refusal, etc.), and incentive group assignment (used to trigger 
incentive mailouts to “unable to locate” and “refusal” cases). 

• Institution comments 

This automated process updated the IMS Web site with searchable case-level 
comments from institution contacting staff. This provided the project team members 
with up-to-date information for use in communicating with institution staff. 

• Master CADE upload 

Each night this process would move CADE data from the public web CADE database 
to the master CADE database inside the RTI firewall. 

® Dataload 

This program contained many different subprocesses, with the overall purpose being 
to process transactions generated during the day by various project systems and 
activities, and post the transactions to the Receipt Control System, updating 
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institution and student-level case status information. Transactions included results 
from enrollment list processing, sampling, CPS matching, CADE preload and data 
receipt processing, lead-letter mailout and return, and CATI/CAPI preloading and 
interviewing. 

• RCS report generator 

Each night following the completion of the dataload process, the RCS report 
generator created HTML pages detailing both the institution- and student-level current 
status reports. It also produced miscellaneous project management reports including: 
Abstraction Method Report, Enrollment (list type) Report, Chief Administrator 
Participation Report, Enrollment List Acquisition Report, CADE Status Summary 
Report (overall and for the B&B cohort), and CATI/CAPI Summary Reports. The 
process automatically posted these reports to the IMS. 
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Chapter 3 
Outcomes of Date Collection 



This chapter presents the overall outcomes of the study procedures described in 
Chapter 2, including institutional participation rates and “yield” rates for each of the sources of 
student data accessed through these procedures. Factors related to these outcomes, including the 
results of planned evaluations, are examined further in subsequent chapters of this report. 

3.1 Institutional Participation 

Only 11 (1 percent) of the 1,083' institutions initially selected for the full-scale study 
were found to be ineligible for NPSAS:2000. The percentage ineligible was substantially less 
than in previous NPSAS rounds because institutions not participating in Title IV aid programs 
were excluded from eligibility in NPSAS:2000. Of the 11 NPSAS-ineligible sampled 
institutions, 7 failed to meet one or more of the NPSAS institutional eligibility criteria specified 
in Chapter 2, 2 closed between the time sampling frame information was collected and 
institutions were first contacted about participation in the study, and 2 were duplicated because of 
mergers with other sampled institutions. Institutional eligibility rates are shown in table 3-1, by 
institutional level of offering, control, and sector. 2 Institutional eligibility varied considerably 
with level of offering and control; it was lowest for less-than-2-year institutions and for the 
private for-profit institutions. These differences were expected, and are consistent with results 
from prior NPSAS rounds. 

The 1,072 eligible sample institutions were asked to participate in NPSAS :2000 by 
(1) providing comprehensive lists of students for sample selection and (2) assisting in abstracting 
data from student records for sampled students. Hence, the potential for institutional 
nonresponse existed at these two points in the survey process. Table 3-1 shows that 999 (93.2 
percent) of the 1,072 eligible sample institutions provided a student enrollment list or database 
that could be used for sample selection. 3 List provision rates (among eligible institutions) varied 
by type of institution considered. 



1 During institutional contacting, it was discovered that part of one institution had recently split off and 
formed a separate institution. Both institutions were considered to be in the sample and therefore increased the 
sample size from 1,082 to 1,083. 

2 In this and subsequent tables, institutional classification errors on the sampling frame were corrected; 
consequently, counts within corrected classifications differ somewhat from those in Chapter 2 based on sampling 
strata. 

3 One institution provided only a baccalaureate list, which was not sufficient for sample selection. 
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Table 3-1. — Overall institutional eligibility and enrollment list participation rates 







Eligible institutions 3 


Institutions providing lists 


Type of institution 1 


Institutions 

sampled 2 


Number 


Percent 4 


Number 


Percent 5 

unweighted 


Percent 5 

weighted 


All institutions 


1,083 


1,072 


99.0 


999 


93.2 


91.3 


Institutional level 














Less-than-2-year 


123 


117 


95.1 


103 


88.0 


87.5 


2-year 


247 


244 


98.8 


232 


95.1 


95.9 


4-year non-doctorate-granting 


317 


315 


99.4 


292 


92.7 


86.1 


4-year doctorate-granting 


396 


396 


100.0 


372 


93.9 


96.2 


Institutional control 














Public 


580 


576 


99.3 


545 


94.6 


94.4 


Private not-for-profit 


376 


371 


98.7 


339 


91.4 


88.6 


Private for-profit 


127 


125 


98.4 


115 


92.0 


91.0 


Institutional sector 














Public less-than-2-year 


34 


32 


94.1 


28 


87.5 


79.1 


Public 2-year 


198 


196 


99.0 


185 


94.4 


96.4 


Public 4-year non-doctorate-granting 


127 


127 


100.0 


123 


96.9 


94.0 


Public 4-year doctorate-granting 


221 


221 


100.0 


209 


94.6 


95.0 


Private not-for-profit 2 -year or less 


35 


32 


91.4 


30 


93.8 


97.6 


Private not-for-profit 4-year non- 


173 


171 


98.8 


153 


89.5 


81.6 


doctorate-granting 














Private not-for-profit 4-year doctorate 


168 


168 


100.0 


156 


92.9 


96.5 


granting 














Private for-profit less-than-2-year 


77 


75 


97.4 


67 


89.3 


' 88.4 


Private for-profit 2 -year or more 


50 


50 


100.0 


48 


96.0 


94.8 



‘institutional classifications were verified by the institutions to correct classification errors on the sampling frame. 

2 During institutional contacting, we discovered that part of one school had recently split off and formed a separate institution. 
Both institutions are included. 

3 Among the 1 1 sampled institutions considered ineligible, 2 had closed since the sampling frame reference period, 2 were 
duplicates with other selected institutions, and the remaining 7 failed to meet one or more of the criteria for institutional NPSAS 
eligibility. 

Percentages are based on the number of institutions sampled within the row under consideration. 

Percentages are based on the number of eligible institutions sampled within the row under consideration. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Postsecondary Student Aid Study, 
1999-2000 (NPSAS: 2000). 

Weighted participation rates were calculated based on the institutional probabilities of 
selection and are also shown in table 3-1 . 4 The overall weighted participation rate of 91 .3 
percent and the weighted rates for most institution categorizations in table 3-1 are similar to the 
unweighted rates. However, NPSAS :2000 was designed to produce efficient estimates only at 
the student level. Institutions were selected with probabilities proportional to size; therefore, 
weighted institution-level estimates are subject to a high level of sampling variation. 



4 The weighted response rates can be interpreted as the estimated percentages of institutions in the 
population that would have provided a usable student sampling list, if asked. 
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3. Outcomes of Data Collection 



3.2 hatching to the Central Processing System 

Table 3-2 summarizes the results of matching and downloading student data from the 
Department of Education’s Central Processing System (CPS). The CPS contains data provided 
to ED by students and their families when they complete the Free Application for Federal 
Student Aid (FAFSA). The matching process required the use of the Federal Data Request 
(FDR) component of ED’s EDConnect software. This component allowed RTI staff to dial into 
the CPS mainframe computer and to upload/download files on a regular basis. Submitting a 
record to the CPS required a valid Social Security number and a valid last name. A successful 
match required that the student have a valid application record within the CPS database. 

The initial CPS matching process occurred after the student sample had been selected for 
an institution, but before institutional record (CADE) data collection activities had begun. This 
matching was against the CPS data for the 1999-2000 financial aid year. As shown in table 3-2, 
not all sample students were submitted to the CPS for matching. This was primarily because 
some institutions were unwilling or unable to provide valid Social Security numbers and last 
names. Following CADE, a small number of student cases that had not previously matched 
successfully to CPS were resubmitted, based on either a newly obtained Social Security number 
or the evidence in the institution records that the student had, in fact, applied for federal student 
aid for the 1999-2000 year. These matching processes included the matching of 1,141 cases to 
the CPS that were subsequently identified as ineligible for NPSAS, because the sample members 
did not meet all of the study eligibility criteria (e.g., not enrolled during the study year). 

As can be seen from table 3-2, the overall matching rate for the 1999-2000 CPS data was 
49 percent. Federal aid applications at public community colleges and technical institutions were 
expected to be proportionately less than in other sectors. Moreover, first-professional students 
tend to rely more on federal aid (primarily loans) whereas graduate students generally rely on 
institutional aid (teaching and research assistantships). 

The NPSAS:2000 sample students were also matched to the 2000-2001 CPS files. It was 
expected that fewer sample students would successfully match to the 2000—2001 CPS files, 
primarily due to students who received degrees or certificates during the 1999-2000 NPSAS year 
and exited postsecondary education. Approximately 500 cases were excluded from matching to 
the 2000-2001 CPS files, because SSNs required for such matching were not available until after 
completion of these activities. Table 3-2 shows that, overall, 52.7 percent of sample students 
matched to either CPS 1999-2000 or CPS 2000-2001, and 25.3 percent matched to both data 
files. 



The proportion of the sample that successfully matched to the CPS 2000-2001 (28.9 
percent) was somewhat lower than the corresponding match rate to CPS 1996—97 obtained 
during the NPSAS:96 study (36.3 percent). This result is not surprising, because the NPSAS:96 
sample included a large number of beginning postsecondary students, who were likely to still be 
enrolled in postsecondary education the following year, whereas the NPSAS :2000 sample 
included a proportionately larger number of baccalaureate recipients, who were more likely to be 
leaving postsecondary education the following year. 



O 

ERIC 



69 



45 



Table 3-2. — Matching sample students to CPS data for 1999-2000 and 2000-2001, by selected institutional and student 
classifications 
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3. Outcomes of Data Collection 



3.3 Abstracting Students’ Institutional Records 

As previously indicated, 999 of the 1,072 eligible sample institutions provided a student 
enrollment list or database that could be used for sample selection. These institutions were 
therefore eligible to participate in the student record abstraction phase of the study referred to as 
CADE (computer-assisted data entry). Table 3-3 shows the weighted and unweighted CADE 
participation rates by several domains of interest. NPSAS:2000 included four CADE abstraction 
methods— Web, data file, field interviewer, and abbreviated CADE — each of which is described 
below. 



At the institution level, an institution was classified as a participating institution if 
sufficient data were obtained for at least one sample student to be classified as a CADE record 
respondent. Only one institution provided CADE data for a single sample member. 

3.3.1 Web-CADE 

Both NPSAS:93 and NPSAS:96 included a computer-based option for NPSAS 
institutions to provide student record data. This has traditionally been known as “self-CADE.” 
For the first time, NPSAS :2000 employed a Web-based methodology for obtaining data from 
student records. Figure 3-1 presents the home page of the NPSAS CADE Web site. As can be 
seen, visitors to the Web site were provided with links to frequently asked questions, information 
about the study, and a mechanism to log into the CADE system. Each Institutional Coordinator 
was mailed a unique CADE identifier, and then was given a password by phone. The login page, 
and all further-nested pages within the CADE application, were protected via a Secure Socket 
Layer (SSL) encryption safeguard. Further security was provided by an automatic “time out” 
feature, through which the user was automatically logged out of the CADE application if the 
system was idle for 20 minutes or longer. The system did not use any persistent “cookies,” thus 
adhering to ED’s privacy policy. Selected Central Processing System (CPS) data were preloaded 
into the web-CADE application before data collection began to reduce data entry burden for 
institution staff. 

In total, 707 of the 999 CADE institutions agreed to provide student data via Web-CADE. 
Ultimately, 694 (74 percent) of the 937 institutions that provided CADE data did so via the 
NPSAS CADE Web site. This proportion was somewhat higher than anticipated, since in 
NPSAS:96, 57 percent of institutions completed “self-CADE.” However, given the availability 
of Web browsers and access to the Internet within the postsecondary education environment, it is 
assumed that the overall familiarity with the Web as a communication medium led to this 
increase. 
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3. Outcomes of Data Collection 



Table 3-3. — Institution-level rates for obtaining institutional record data (CADE), 



by institutional sector and method of data a 


^fraction 








Number of 


Institutions providing CADE 2 


Type of institution 1 


institutions 

providing 

lists 


Number 


Unweighted 

percent 3 


Weighted 

percent 3 


All institutions 


999 


937 


93.8 


95.0 


Institution level 










Less-than-2-year 


103 


89 


86.4 


91.1 


2-year 


232 


222 


95.7 


98.6 


4-year non-doctorate-granting 


292 


274 


93.8 


94.6 


4-year doctorate-granting 


372 


352 


94.6 


94.1 


Institutional control 










Public 


545 


514 


94.3 


95.8 


Private not-for-profit 


339 


317 


93.5 


93.6 


Private for-profit 


115 


106 


92.2 


95.6 


Institutional sector 










Public less-than-2-year 


28 


25 


89.3 


88.2 


Public 2 -year 


185 


176 


95.1 


97.3 


Public 4-year non-doctorate-granting 


123 


117 


95.1 


95.4 


Public 4-year doctorate-granting 


209 


196 


93.8 


93.8 


Private not-for-profit 2-year or less 


30 


27 


90.0 


93.8 


Private not-for-profit 4-year non-doctorate-granting 


153 


141 


92.2 


93.3 


Private not-for-profit 4-year doctorate-granting 


156 


149 


95.5 


94.0 


Private for-profit less-than-2-year 


67 


58 


86.6 


92.4 


Private for profit 2 -year or more 


48 


48 


100.0 


100.0 


Abstraction method 










Web 


707 


694 


98.2 


97.5 


Data file 


71 


29 


40.8 


33.0 


Field interviewer 


221 


214 


96.8 


98.9 



Institutional classifications were verified by participating institutions to correct classification errors on the sampling frame. 
Provided institutional record data for at least one sampled student. 

Percentages are based on the number of eligible institutions that provided a list for sampling. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Postsecondary Student Aid Study 
1999-2000 (NPSAS:2000). 
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3. Outcomes of Data Collection 



Figure 3-1. — NPSAS CADE home page 
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7n a tip na I Postsoconda ry Stud© n tAldStucI y 

The data collection for the NPSAS period starting from July 
1999 to June 2000 has ended. Thank you for your 
participation. 

Welcome to the NPSAS:2000 CADE Website! 

This site has been designed for use during the Institution Data 
Collection phase of NPSAS: 2000. This phase of the study is known 
as CADE because it includes computer-assisted data entry as a 
means of data collection. This site provides access to information 
about the study, instructions for participating institutions, and 
answers to Frequently Asked Questions. More importantly, it is 
through this site that institutions will connect to the CADE 
database in order to provide information about NPSAS sample 
students. 

Privacy Policy and Use of Cookies by the NPSAS:2000 web 
site. 

This notice is to assure respondents that NPSAS does not use 
cookies to collect personal information. This system was 
developed using Allaire’s ColdFusion Pages. This technology (and 
competing technologies) requires the web server to remember 
users as they move from page to page during their data entry 
session. The web server accomplishes this task b y assig ning a 






M 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Postsecondary Student Aid Study, 
1999-2000 (NPSAS:2000). 



3.3.2 Datafile-CADE 

As an alternative to keying data into the Web-CADE application, institutions were given 
detailed specifications for developing a set of data files containing student record data. Twenty- 
nine institutions, predominantly 4-year institutions, opted for this method of CADE abstraction. 
The specifications were customized for each institution so that they would have their own coding 
schemes for reporting various types of institution and state aid (the names of which were 
obtained from the Institutional Coordinator during the institution contacting phase of the study). 
Eight data files, including student-level, term-level, and aid award-level files, were required from 
each datafile-CADE institution in order to accurately match the identical data structure of the 
database underlying the Web-CADE application. Upon completion of the datafile-CADE file 
preparation, institutions submitted their data files back to RTI via the Web-CADE application. 
Upon submission, an automated quality control system processed the files and instantly reported 
back to the institutions any anomalies in the data (e.g., incorrect student ID variables, lack of 
term-level data for sample students, incorrect file names, etc.). 
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3. Outcomes of Data Collection 



3.3.3 Field-CADE 

Consistent with procedures implemented in both NPSAS:93 and NPSAS:96, institutions 
were given the option of having an RTI-employed field data collector visit the institution and 
provide student record data-entry services at no expense to the institution. This CADE 
abstraction method is referred to as field-CADE. In total, 214 institutions opted for field-CADE. 
In most instances, field data collectors were able to complete the data collection activities in 1 
week or less, although certain institutions with a relatively high number of sample students 
required as much as 2 weeks of field data collector activity to complete the collection. 

Field data collectors used a laptop-based CADE system for entering data abstracted from 
student records. The system included real-time edit features to help detect out-of-range or 
inconsistent entries. Data previously obtained from the Central Processing System were preload- 
ed into the system before data collection began, to reduce the data collectors’ level of effort. 

3.3.4 Abbreviated CADE 

A fourth method of CADE abstraction was used for the first time in NPSAS:2000. This 
procedure, known as “abbreviated CADE,” was intended as a last-ditch effort to obtain 
participation by sample institutions. Essentially, institutions that had not provided an enrollment 
list by late fall of 2000 were given the option of being excluded from the separate, complete 
CADE process. Instead, they were allowed to provide an enhanced enrollment list containing not 
only the data necessary for sampling, but also selected student attributes and locating data. 5 This 
set of 17 variables was considered sufficient for use in initializing the telephone interviewing 
system for the sample students, thus providing an opportunity to interview the students. These 
data were considered insufficient for defining the student-level case as a CADE respondent. 
Although not shown in table 3-3, 40 institutions chose to participate in this manner; these 
instructions are included in the “data file” count. 

Rates for obtaining CADE data for the NPSAS:2000 sample students are shown in 
table 3-4. Again, both weighted and unweighted results are shown. The CADE data collection 
phase of the study was restricted to those students enrolled in the institutions providing an 
enrollment list from which a student sample could be selected. About 5,800 of the 70,200 
sampled were subsequently determined not to meet the study eligibility requirements. Hence, the 
eligible CADE student sample consisted of about 64,500 students. 



s The 17 variables requested on the enrollment list for purposes of classifying an institution as participating 
in the study included student SSN, first name, last name, middle initial, student level, local phone and address 
variables, permanent phone and address variables, IPEDS ID of school attended, bachelor’s degree recipient status, 
total institution grant aid, total state grant aid, and student major/field of study. 
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3. Outcomes of Data Collection 



Table 3-4. — Student-level rates for obtaining institutional record data (CADE), by 
selected institutional and student classifications 





Number 


CADE completion 


Type of student 1 


of eligible 
students 2 


Number 3 


Unweighted 

percent 4 


Weighted 

percent 4 




64,471 


59,290 


92 


97 


All students 
Institution level 






Less-than-2-year 


5,810 


4,640 


80 


86 


2-year 


11,548 


10,970 


95 


95 


4-year non-doctorate-granting 


17,383 


16,280 


94 


92 


4-year doctorate-granting 


29,730 


27,400 


92 


94 


Institutional control 










Public 


39,984 


37,200 


93 


93 


Private not-for-profit 


17,995 


16,440 


91 


94 


Private for-profit 


6,492 


5,650 


87 


94 


Institutional sector 










Public less-than-2-year 


1,169 


910 


78 


82 


Public 2-year 


9,167 


8,690 


95 , 


94 


Public 4-year non-doctorate -granting 


9,132 


8,680 


95 


91 


Public 4-year doctorate-granting 


20,516 


18,920 


92 


93 


Private not-for-profit 2-year or less 


1,659 


1,480 


89 


97 


Private not-for-profit 4-year non-doctorate-granting 


7,481 


6,840 


91 


92 


Private not-for-profit 4-year doctorate-granting 


8,855 


8,130 


92 


95 


Private for-profit less-than-2-year 


4,096 


3,320 


81 


87 


Private for profit 2 -year or more 


2,396 


2,330 


97 


98 


Student level 










Undergraduate 


52,033 


48,010 


92 


94 


Graduate 


11,155 


10,150 


91 


92 


First-professional 


1,283 


1,130 


88 


91 


Abstraction method 5 










Web 


42,421 


41,130 


97 


99 


Data file 5 


3,592 


2,940 


82 


86 


Field interviewer 


16,016 


15,210 


95 


96 



'institutional classifications were verified by participating institutions to correct classification errors on the sampling frame, 
includes all 70,232 sampled students minus the 5,761 found to be NPSAS-ineligible at any stage of data collection. 

3 A student was classified as a CADE record completion if key demographic, enrollment, and financial aid data were provided. 
Percentages are based on eligible students within the row under consideration. 

Excludes 2,442 students with abbreviated CADE information. 

NOTE: To protect confidentiality, some numbers have been rounded. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Postsecondary Student 
Aid Study, 1999-2000 (NPSAS:2000). 



3. Outcomes of Data Collection 



A student record was considered to represent a CADE record respondent if it met the 
following criteria: 

• the CADE financial aid gate question was answered (Yes or No, including derived 
answer from abbreviated CADE cases), AND 

• some amount of CADE enrollment data was provided (as indicated by at least one of 
the 12 monthly enrollment indicators being nonzero), AND 

• the CADE student characteristics section had at least one valid response for the set of 
items (date of birth, marital status, race, sex). If the case was a CPS match, it was 
considered to have successfully met this criterion. 

Overall, the unweighted CADE student record response rate (the percentage of study- 
eligible cases for whom a sufficiently complete CADE record was obtained) was 92 percent. The 
rate was lowest among students from public less-than-2-year institutions (78 percent) and highest 
among students from private for-profit 2-year-or-more institutions (97 percent). As was 
previously mentioned, institutions classified as abstracted through abbreviated CADE did not 
actually complete the record abstraction process. Rather, these institutions provided a more 
thorough set of data as part of the enrollment list. However, this set of 17 variables was not 
considered sufficient for a student to be considered a CADE record respondent. 

3.4 Matching to NSLDS for Loan and Grant Data 

Results of the National Student Loan Data System (NSLDS) attempted loan matching are 
shown in table 3-5. Because NSLDS files are historical, information about receipt of such loans 
was available not only for the NPSAS year but also for prior years of postsecondary education 
(where applicable); therefore the table shows match rates for both the NPSAS year and 
historically. In total, 21,410 study respondents (34.9 percent of those submitted) were matched 
for the NPSAS year. This is consistent with the NPSAS :96 result of 34.2 percent. Over all 
years, 34,089 study respondents (55.6 percent) were matched, including both undergraduate and 
graduate students. 
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3. Outcomes of Data Collection 



Table 3-5. — Results off NSLDS loan matching, by selected institutional and student 







Matched to loan data 3 




Type of student 1 


Study 


Loan during 
NPSAS year 


Loan historically 4 




respondents 2 


Number 


Percent 


Number 


Percent 




61,330 


21,410 


35 


34,090 


56 


All students 
Institution level 








Less-than-2-year 


5,080 


1,840 


36 


2,860 


56 


2 -year 


11,150 


2,320 


21 1 


4,460 


40 


4-year non-doctorate-granting 


16,760 


6,700 


40 


10,170 


61 


4-year doctorate -granting 


28,340 


10,550 


37 


16,590 


59 


Institutional control 












Public 


38,570 


11,140 


29 


19,300 


50 


Private not-for-profit 


16,910 


7,330 


43 


10,630 


63 


Private for-profit 


5,850 


2,940 


50 


4,160 


71 


Institutional sector 










27 


Public less-than-2-year 


1,050 


90 


8 


280 


Public 2-year 


8,910 


1,030 


12 


2,810 


32 


Public 4-year non-doctorate -granting 


8,940 


3,040 


34 


5,010 


56 


Public 4-year doctorate-granting 


19,680 


6,990 


36 


11,200 


57 


Private not-for-profit 2-year or less 


1,470 


630 


43 


860 


58 


Private not-for-profit 4-year non-doctorate -granting 


7,130 


3,260 


46 


4,640 


65 


Private not-for-profit 4-year doctorate-granting 


8,320 


3,440 


41 


5,130 


62 


Private for-profit less-than-2-year 


3,590 


1,600 


45 


2,360 


66 


Private for profit 2-year or more 


2,260 


1,340 


59 


1,800 


80 


Student level 












Undergraduate 


49,620 


18,140 


37 


27,360 


55 


Graduate 


10,510 


2,430 


23 


5,760 


55 


First-professional 


1,200 


840 


70 


980 


81 



'Both institutional and student classifications were verified to correct classification errors on the sampling frame, 
includes study respondents for whom an apparently legitimate Social Security number was available. Study respondents were 
defined as eligible sample students for whom completed CADE and/or student interview data were obtained. 

3 The loan transaction matches for any year do not necessarily reflect a loan during the year. They may represent a consolidation 
or cancellation transaction. 

4 Over all years of postsecondary education reflected in the NSLDS files. 

NOTE: To protect confidentiality, some numbers have been rounded. All percentages are unweighted and based on 
the total number of study respondents within the row under consideration. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Postsecondary Student Aid Study, 
1999-2000 (NPSAS:2000). 
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3. Outcomes of Data Collection 



For NSLDS matches for the NPSAS year and within the student classifications 
considered, the relative numbers of matches followed a pattern quite similar to that seen for the 
CPS matching. The table shows low match rates for graduate students and for those in public 
institutions with program offerings of 2 years or less, but high match rates for first professional 
students and those in private for-profit institutions. This was not surprising given the expectation 
that federal aid applications at public community colleges and technical institutions would be 
less than for other types of institutions. In addition, graduate students generally depend on 
institutional aid such as assistantships, while first-professional students tend to depend primarily 
on federal loans. 

Results of attempted matches to the NSLDS Pell grant data are shown in table 3-6. 
Matches were obtained for 13,500 study respondents (22 percent of those submitted) for the 
NPSAS year. 

Table 3-6. Results of NSLDS Pell grant matching, by selected institutional and student 
classifications 



Type of student 1 






Matched to Pell data 




Study 


Grants during 


NPSAS year 


Grant historically 3 




respondents 2 


Number 


Percent 


Number 


Percent 


All students 


61,330 


13,550 


22 


21,430 


35 


Institution level 












Less-than-2-year 


5,080 


2,670 


52 


3,180 


63 


2-year 


11,150 


2,750 


25 


4,090 


37 


4-year non-doctorate-granting 


16,760 


3,990 


24 


6,370 


38 


4-year doctorate-granting 
Institutional control 


28,340 


4,150 


15 


7,790 


28 


Public 


38,570 


7,320 


19 


12,510 


32 


Private not-for-profit 


16,910 


3,360 


20 


5,390 


32 


Private for-profit 


5,850 


3,870 


66 


3,520 


60 


Institutional sector 






Public less-than-2-year 


1,050 


280 


27 


390 


37 


Public 2-year 


8,910 


1,760 


20 


2,830 


32 


Public 4-year non-doctorate-granting 


8,940 


2,070 


23 


3,430 


38 


Public 4-year doctorate-granting 


19,680 


3,210 


16 


5,870 


30 


Private not-for-profit 2-year or less 


1,470 


690 


47 


880 


60 


Private not-for-profit 4-year non-doctorate- 


7,130 


1,740 


24 


2,670 


38 


granting 








Private not-for-profit 4-year doctorate granting 


8,320 


920 


11 


1,840 


22 


Private for-profit less-than-2-year 


3,590 


2,090 


58 


2,440 


68 


Private for profit 2-year or more 


2,260 


780 


35 


1,080 


48 


Student level 








Undergraduate 


49,620 


13,490 


27 


19,750 


40 


Graduate 


10,510 


60 


1 


1,410 


13 


First-professional 


1,200 


10 


1 


260 


22 



^Both institutional and student classifications were verified to correct classification errors on the sampling frame. 

Includes study respondents for whom an apparently legitimate Social Security number was available. Study respondents were 
defined as eligible sample students for whom completed CADE and/or student interview data were obtained. 

3 Over all years of postsecondary education reflected in the NSLDS files. 

NOTE: To protect confidentiality, some numbers have been rounded. All percentages are unweighted and based on 
the total number of study respondents within the row under consideration. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Postsecondary Student Aid Studv 
1 999-2000 (NPSAS:2000). 
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3. Outcomes of Data Collection 



This is consistent with the NPSAS:96 result of 22 percent. Over all years, 21,400 study 
respondents (35 percent) were matched. A handful of the matches for the NPSAS year involved 
graduate and first-professional students, who were not eligible for this form of financial aid. 
However, the matched graduate and first-professional sample members were undergraduates at 
some time during the year (and as such were eligible for this type of aid during the year). 
Consistent with expectations, the Pell match rate was highest among students at private for-profit 
less-than-2-year institutions. 

3.5 Matching to ACT and SAT Data 

Interview data pertaining to standard test scores have typically been characterized by high 
rates of nonresponse. To overcome this problem and provide this additional information to the 
student characteristics profile, student records were matched to the ACT and SAT files. Results 
of the ACT and SAT score matching are shown in table 3-7. A total of 16,500 unique cases 
matched to ACT data in the years 1991-92 through 1999-2000 (27 percent). If a student 
matched to more than one year, only the most recent test year information was kept on the file. 
SAT matches were acquired for 14,700 of the respondent cases (24 percent). This matching was 
conducted for test years 1995 through 1999. Similar to the ACT, if a student matched to more 
than one SAT test year, only the most recent record was kept in the file. 

The highest rate of matches to the ACT file occurred with the public, 4-year institutions. 
These are the types of institutions that typically require the ACT, particularly in the middle part 
of the country. Students from schools with program offerings of 2 years or less experienced the 
lowest match rates. These students usually do not need to take the ACT. Another difference in 
match rates occurred among student levels. The graduate student match rate was much lower 
than the first-professional rate, and one would expect these to be comparable. This may be 
explained by looking at the average student age within the student levels. The first-professional 
average age was 27.8 years, while the graduate average age was 33.4 years. The graduate 
students were, on average, 5.6 years older than the first-professional students were. Therefore, 
the much lower rate for graduate students probably occurred because the matches of graduate test 
records did not extend far enough back in time to capture them. 

The highest match rate to the SAT file was for students at schools with program offerings 
of 4 years. The rates were lowest for the 2-year-or-less institutions. In addition, rates were fairly 
low for the private for-profit schools. Consistent with the ACT matches, these rates reflect the 
type of institutions requiring the SAT. The low graduate and first-professional rates (as well as 
the difference between those two) can probably be explained by the average age differences 
among the different student levels, as described in the ACT discussion above. 

3.6 Student Locating and Interviewing 

Collecting data directly from student sample members in NPSAS :2000 consisted of three 
sequential steps: locating (identifying an initial telephone number or address at which the sample 
member could be reached), contacting (making the necessary attempts to reach the sample 
member), and interviewing (convincing the sample member to cooperate and participate in the 
interview). The amount of time and level of effort required to complete these steps with any 
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3. Outcomes of Data Collection 



given sample member varied considerably. Some sample members were reached and interviewed 
on the first attempt at contact. Others required considerable tracing (contacting of parents, former 
roommates, etc.) before they were successfully located and interviewed. Student interviewing for 
NPSAS was also complicated by the two-tiered study design (separate institutional and student 
data collections) and the varying rates of cooperation at the institution level. As a result, not all 
cases were available to be worked at the start of CATI data collection. Rather, the cases flowed 
into CATI after student lists were obtained from schools, students were sampled from the lists, 
and CADE information (particularly locating information) was collected from the participating 
institutions. 

Figure 3-2 illustrates outcomes of student locating and interviewing and related case- 
resolution activities. Student data were collected primarily by computer-assisted telephone 
interview (CATI), with follow-up of nonrespondents by computer-assisted personal interview 
(CAPI) and/or self-administered mail survey. The data collection period ran from May 22, 2000 
to February 28, 2001. 

One week before a student case was released into CATI production, sample members 
were sent an advance mailing, which included a cover letter and study leaflet. These letters were 
mailed in batches twice a week as new cases were loaded into CATI to be worked. Letters were 
mailed to 64,800 sample. Additionally, 6,300 sample members requested that a letter be 
remailed during data collection, because they had either misplaced the letter or not received it. 

Attempts were made to locate 66,300 of the original 70,200 sampled (3,300 cases were 
determined to be ineligible for NPSAS during CADE and 640 were sampled but not loaded into 
CATI because they had no locating/tracing information and/or such information was obtained too 
late). Overall, 54,400 (82 percent), including CATI ineligibles and exclusions, of the initial 
CATI sample were located; 12,000 (18 percent) of the original sample were not located. Of 
those located, 44,500 completed all or part of the interview; 6,500 were located, but did not 
complete the interview; 2,500 were determined to be ineligible for NPSAS based on their 
responses to the interview; and, about 900 were considered exclusion cases. 6 

Student interviewing results for those students who were located are also shown 
schematically in figure 3-2. Approximately 40,400 completed the entire interview, while 3,300 
completed either a paper-copy mail questionnaire or an “abbreviated” interview (that is, a version 
of the questionnaire containing key data elements), and 750 completed only part (including at 
least section A) of the NPSAS interview. 7 



Exclusion cases consisted of students who were out of the country, unavailable during survey period, 
institutionalized, incapacitated or who had a language barrier. 

A large percentage (2,450 of 3,300) of the “abbreviated” interviews were conducted with Spanish- 
speaking-only sample members. 



Table 3-7. — Results of ACT and SAT score matching, by selected institutional and student classifications 
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3. Outcomes of Data Collection 



Figure 3-2. — Student sample case flow through locating, CATI interviewing, and related 
case resolution 




SOURCE: U.S. Department of Education, National Center for Education Statistics, National Postsecondary Student Aid Study 
1999-2000 (NPSAS:2000). . y ’ 
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3. Outcomes of Data Collection 



A total of 6,500 potentially eligible students who were located were not interviewed. Of these, 
about 5,200 were explicit final refusals. These cases represent situations in which subsequent 
attempts at interviewing were determined to be infeasible or unwise. Also not interviewed were 
1 ,340 sample members for whom time ran out before they could complete the interview. These 

cases were loaded late in the data collection period (in January or February 2001), restricting the 

8 

time to adequately work them. 

NPSAS:2000 student locating and interviewing (for those located) results by institution 
type and student type are provided in table 3-8, for eligible sample members for whom CATI 
locating was attempted. Students in private for-profit institutions proved to be more difficult to 
find (locate rates: 72 percent private for-profit; 82 percent private not-for-profit; 82 percent 
public) and slightly less willing to participate once the student was located (interviewed-when- 
located rates: 85 percent private for-profit; 88 percent private not-for-profit; 87 percent public). 
Similarly, the locate rates were lower for students in less-than-2-year schools (71 percent) and 2- 
year institutions (78 percent) than they were for either 4-year doctorate-granting (88 percent) or 
4-year non-doctorate-granting (89 percent) institutions. In terms of student type, baccalaureate 
recipients (84 percent) and graduate and first-professional students (83 percent) were easier to 
locate than were non-baccalaureate-receiving undergraduates (79 percent). Once they were 
located, however, there were only slight differences among these groups in terms of the 
percentage interviewed. 

Weighted overall CATI response rates are provided in table 3-9 and constitute the target 
population directly represented by the NPSAS:2000 study respondents. This rate was computed 
as the product of the weighted institution and student response rates. Coverage of entire clusters 
of students was lost when sample institutions did not participate. Additionally, coverage was lost 
when individual students in participating institutions failed to respond. The cumulative effect on 
coverage of the student population is reflected by the overall weighted student CATI response 
rate of 66 percent, ranging from 72 percent for students attending private, not-for-profit, 
doctorate-granting institutions to 57 percent for students attending public, less-than-2-year 
institutions. 

3.7 Overall Study Participation 

The students included in the final NPSAS:2000 analysis database were defined to be the 
overall “study respondents,” meeting the requirements specified above for being a CADE record 
respondent and/or CATI respondent. Using this definition of the overall study response status, 
table 3-10 shows that about 62,000 of the 64,500 eligible sample students were classified as 
“study respondents” for an unweighted study response rate of 96 percent. This table also 
presents the study response rates, weighted and unweighted, by various institutional and student 
classifications. The weighted rates are based on the student sampling weights with adjustments 
for institutional nonresponse and for student multiplicity (attendance at more than one NPSAS- 
eligible institution during the NPSAS year). The overall weighted study response rate in table 3- 



8 This group likely contains, however, an unknown number of implicit refusal cases, individuals who after 
first contact used answering machines or friends/relatives as gatekeepers, as well as those who continued to make 
(and then break) appointments for an interview. 



3. Outcomes of Data Collection 



10 was 89 percent. Both weighted and unweighted response rates shown in table 3-10 are quite 
consistent. 

Table 3-8. — NPSAS:2000 student locating and interview results by institution and student 

type 



Type of student 1 




Located 


Interviewed when located 


Total 2 


Number 

located 


Percent 

located 


Number 

interviewed 


Percent 

interviewed 


All students 


62,970 


51,010 


81 


44,490 


87 


Institution level 












Less-than-2-year 


5,560 


3,940 


71 


3,360 


85 


2-year 


11,350 


8,890 


78 


7,490 


84 


4-year non-doctorate-granting 


17,090 


14,280 


84 


12,630 


89 


4-year doctorate -granting 


28,960 


23,900 


83 


21,020 


88 


Institutional control 










Public 


39,330 


32,250 


82 


28,060 


87 


Private not-for-profit 


17,340 


14,200 


82 


12,540 


88 


Private for-profit 


6,300 


4,560 


72 


3,890 


85 


Institutional sector 










Public less-than-2-year 


1,150 


870 


76 


740 


86 


Public 2 -year 


9,050 


7,130 


79 


5,950 


84 


Public 4-year non-doctorate-granting 


9,040 


7,620 


84 


6,730 


88 


Public 4-year doctorate -granting 


20,090 


16,630 


83 


14,640 


88 


Private not-for-profit 2-year or less 


1,530 


1,110 


73 


980 


88 


Private not- 4-year non-doctorate-granting 


7,290 


6,090 


84 


5,410 


89 


Private not-for-profit 4-year doctorate- 
granting 


8,520 


7,000 


82 


6,150 


88 


Private for-profit less-than-2-year 


3,940 


2,760 


70 


2,350 


85 


Private for-profit 2-year or more 


2,360 


1,800 


76 


1,550 


86 


Student type 








Total undergraduate 


50,840 


40,890 


80 


35,540 


87 


B&B 


14,030 


11,780 


84 


10,400 


88 


Other undergraduates 


36,810 


29,110 


79 


25,130 


86 


Graduate 


10,870 


9,080 


84 


8,040 


89 


First-professional 


1,250 


1,040 


83 


920 


88 



Both institution and student classifications were verified to correct classification errors on the sampling frame. 

Statistics exclude 5,761 NPSAS-ineligible sample members (as determined during record extraction or in CATI); 868 sample 
members who were either unavailable for the duration of the survey, out of country, or institutionalized; and 638 cases that were 
sampled but never worked in CATI. 

NOTE: To protect confidentiality, some numbers have been rounded. All percentages are unweighted and based on the eligible 
count within the row under consideration. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Postsecondary Student Aid Study 
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Table 3-9.— Student interview (CATI) response rates, by selected institutional and student classifications 
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Table 3-10. — Study respondents, by selected institutional and student classifications 
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The weighted institution response rate for a given student level was calculated as the response rate of all institutions with that level of offering. 

NOTE: To protect confidentiality, some numbers have been rounded. Percentages are based on the eligible students within the row under consideration. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Postsecondary Student Aid Study, 1999-2000 (NPSAS:2000). 



